2020-05-18T21:10:57.255 INFO:root:teuthology version: 1.0.0-f81437e 2020-05-18T21:10:57.257 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_ffsb.yaml} --archive /home/teuthworker/archive/pdonnell-2020-05-18_20:49:47-fs-wip-pdonnell-testing-20200516.030215-distro-basic-smithi/5067934 --name pdonnell-2020-05-18_20:49:47-fs-wip-pdonnell-testing-20200516.030215-distro-basic-smithi --owner scheduled_pdonnell@teuthology --verbose --lock --block -- /tmp/teuthology-worker.4lzuisq_.tmp 2020-05-18T21:10:57.292 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-05-18T21:10:57.352 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2020-05-18_20:49:47-fs-wip-pdonnell-testing-20200516.030215-distro-basic-smithi/5067934 branch: wip-pdonnell-testing-20200516.030215 description: fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_ffsb.yaml} email: pdonnell@redhat.com first_in_suite: false job_id: '5067934' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2020-05-18_20:49:47-fs-wip-pdonnell-testing-20200516.030215-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-pdonnell-testing-20200516.030215 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd_max_omap_entries_per_request: 10000 fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - SLOW_OPS - slow request sha1: 543315c934420269aa12ef2f9dec2c9eadb4fa6f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: 543315c934420269aa12ef2f9dec2c9eadb4fa6f thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20200516.030215 sha1: 543315c934420269aa12ef2f9dec2c9eadb4fa6f owner: scheduled_pdonnell@teuthology priority: 75 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 sha1: 543315c934420269aa12ef2f9dec2c9eadb4fa6f sleep_before_teardown: 0 suite: fs suite_branch: wip-pdonnell-testing-20200516.030215 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200516.030215/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 543315c934420269aa12ef2f9dec2c9eadb4fa6f tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - check-counter: counters: mds: - mds.dir_split - workunit: clients: all: - suites/ffsb.sh teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.30699 2020-05-18T21:10:57.352 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200516.030215/qa; will attempt to use it 2020-05-18T21:10:57.353 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200516.030215/qa/tasks 2020-05-18T21:10:57.353 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-05-18T21:10:57.355 INFO:teuthology.task.internal:Checking packages... 2020-05-18T21:10:57.386 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash '543315c934420269aa12ef2f9dec2c9eadb4fa6f' 2020-05-18T21:10:57.386 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-05-18T21:10:57.387 INFO:teuthology.packaging:ref: None 2020-05-18T21:10:57.387 INFO:teuthology.packaging:tag: None 2020-05-18T21:10:57.388 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20200516.030215 2020-05-18T21:10:57.388 INFO:teuthology.packaging:sha1: 543315c934420269aa12ef2f9dec2c9eadb4fa6f 2020-05-18T21:10:57.388 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-pdonnell-testing-20200516.030215 2020-05-18T21:10:57.640 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-1640-g543315c9344-1focal 2020-05-18T21:10:57.641 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-05-18T21:10:57.650 INFO:teuthology.task.internal:no buildpackages task found 2020-05-18T21:10:57.651 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-05-18T21:10:57.686 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-05-18T21:10:57.686 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-05-18T21:10:57.751 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:11:07.773 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:11:17.796 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-18T21:11:27.816 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-18T21:11:37.849 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-18T21:11:47.864 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-18T21:11:57.877 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-18T21:12:07.899 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-18T21:12:17.916 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-18T21:12:27.934 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-18T21:12:37.949 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-18T21:12:47.967 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-18T21:12:57.988 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:13:08.011 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:13:18.028 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:13:28.048 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:13:38.065 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:13:48.086 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:13:58.104 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:14:08.123 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:14:18.136 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:14:28.155 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:14:38.171 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:14:48.192 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:14:58.213 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:15:08.231 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:15:18.244 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:15:28.263 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:15:38.277 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:15:48.296 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:15:58.313 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:16:08.331 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:16:18.349 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:16:28.365 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:16:38.383 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:16:48.399 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:16:58.418 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:17:08.435 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:17:18.447 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:17:28.464 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:17:38.479 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:17:48.500 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:17:58.519 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:18:08.532 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:18:18.548 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:18:28.562 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:18:38.573 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:18:48.593 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:18:58.612 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:19:08.637 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:19:18.661 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:19:28.680 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:19:38.702 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:19:48.722 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:19:58.743 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:20:08.764 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:20:18.794 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:20:28.813 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:20:38.833 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:20:48.854 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:20:58.872 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:21:08.895 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:21:18.916 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:21:28.935 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:21:38.952 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:21:48.973 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:21:58.993 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:22:09.015 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:22:19.042 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:22:29.063 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:22:39.079 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:22:49.115 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:22:59.135 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:23:09.152 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:23:19.172 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:23:29.191 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:23:39.209 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:23:49.229 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:23:59.248 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:24:09.269 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-18T21:24:19.290 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:24:29.311 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:24:39.333 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:24:49.352 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:24:59.373 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-18T21:25:09.392 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_pdonnell@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/pdonnell-2020-05-18_20:49:47-fs-wip-pdonnell-testing-20200516.030215-distro-basic-smithi/5067934'} 2020-05-18T21:25:09.430 DEBUG:teuthology.lock.ops:locked ubuntu@smithi196.front.sepia.ceph.com, ubuntu@smithi080.front.sepia.ceph.com 2020-05-18T21:25:09.497 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa270992ae8>, signals=[15]) 2020-05-18T21:25:09.721 INFO:teuthology.provision.fog.smithi196:Scheduling deploy of ubuntu 20.04 2020-05-18T21:25:09.727 INFO:teuthology.provision.fog.smithi080:Scheduling deploy of ubuntu 20.04 2020-05-18T21:25:10.127 INFO:teuthology.orchestra.console:Power off smithi080 2020-05-18T21:25:10.129 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-05-18T21:25:10.166 INFO:teuthology.orchestra.console:Power off smithi196 2020-05-18T21:25:10.168 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-05-18T21:25:10.180 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-18T21:25:10.194 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-18T21:25:14.214 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-18T21:25:14.229 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-18T21:25:18.334 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-18T21:25:18.349 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-18T21:25:22.454 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-18T21:25:22.467 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-18T21:25:22.668 INFO:teuthology.orchestra.console:Power off for smithi080 completed 2020-05-18T21:25:22.686 INFO:teuthology.orchestra.console:Power off for smithi196 completed 2020-05-18T21:25:22.770 INFO:teuthology.orchestra.console:Power on smithi080 2020-05-18T21:25:22.771 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-05-18T21:25:22.796 INFO:teuthology.orchestra.console:Power on smithi196 2020-05-18T21:25:22.798 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-05-18T21:25:22.809 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-18T21:25:22.829 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-18T21:25:26.846 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-18T21:25:26.858 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-18T21:25:27.061 INFO:teuthology.orchestra.console:Power on for smithi080 completed 2020-05-18T21:25:27.070 INFO:teuthology.orchestra.console:Power on for smithi196 completed 2020-05-18T21:25:27.162 INFO:teuthology.provision.fog.smithi080:Waiting for deploy to finish 2020-05-18T21:25:27.171 INFO:teuthology.provision.fog.smithi196:Waiting for deploy to finish 2020-05-18T21:28:28.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:28:28.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:28:53.073 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:28:53.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:29:02.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:29:02.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:29:11.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:29:11.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:29:20.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:29:20.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:29:29.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:29:57.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:03.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:06.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:09.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:12.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:15.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:18.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:22.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:23.015 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-05-18T21:30:23.018 INFO:teuthology.orchestra.run.smithi196:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-05-18T21:30:23.545 INFO:teuthology.provision.fog.smithi196:Node is ready 2020-05-18T21:30:23.546 INFO:teuthology.orchestra.run.smithi196:> hostname 2020-05-18T21:30:23.600 INFO:teuthology.orchestra.run.smithi196.stdout:smithi196.front.sepia.ceph.com 2020-05-18T21:30:23.601 INFO:teuthology.orchestra.run.smithi196:> grep smithi196.front.sepia.ceph.com /etc/hosts 2020-05-18T21:30:23.608 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.196 smithi196.front.sepia.ceph.com smithi196 2020-05-18T21:30:23.608 INFO:teuthology.orchestra.run.smithi196:> sudo hostname smithi196 2020-05-18T21:30:23.766 INFO:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/smithi196.front.sepia.ceph.com/smithi196/g' /etc/hosts 2020-05-18T21:30:23.785 INFO:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/172.21.15.196/172.21.15.196/g' /etc/hosts 2020-05-18T21:30:23.797 INFO:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/smithi196.front.sepia.ceph.com/smithi196/g' /etc/hostname 2020-05-18T21:30:23.870 INFO:teuthology.orchestra.run.smithi196:> sudo hostname smithi196 2020-05-18T21:30:23.887 INFO:teuthology.provision.fog.smithi196:Deploy complete! 2020-05-18T21:30:23.887 DEBUG:teuthology.parallel:result is None 2020-05-18T21:30:24.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:30.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:30.977 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-05-18T21:30:30.978 INFO:teuthology.orchestra.run.smithi080:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-05-18T21:30:31.582 INFO:teuthology.provision.fog.smithi080:Node is ready 2020-05-18T21:30:31.583 INFO:teuthology.orchestra.run.smithi080:> hostname 2020-05-18T21:30:31.589 INFO:teuthology.orchestra.run.smithi080.stdout:smithi080.front.sepia.ceph.com 2020-05-18T21:30:31.590 INFO:teuthology.orchestra.run.smithi080:> grep smithi080.front.sepia.ceph.com /etc/hosts 2020-05-18T21:30:31.597 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80 smithi080.front.sepia.ceph.com smithi080 2020-05-18T21:30:31.598 INFO:teuthology.orchestra.run.smithi080:> sudo hostname smithi080 2020-05-18T21:30:31.761 INFO:teuthology.orchestra.run.smithi080:> sudo sed -i -e 's/smithi080.front.sepia.ceph.com/smithi080/g' /etc/hosts 2020-05-18T21:30:31.776 INFO:teuthology.orchestra.run.smithi080:> sudo sed -i -e 's/172.21.15.80/172.21.15.80/g' /etc/hosts 2020-05-18T21:30:31.837 INFO:teuthology.orchestra.run.smithi080:> sudo sed -i -e 's/smithi080.front.sepia.ceph.com/smithi080/g' /etc/hostname 2020-05-18T21:30:31.877 INFO:teuthology.orchestra.run.smithi080:> sudo hostname smithi080 2020-05-18T21:30:31.892 INFO:teuthology.provision.fog.smithi080:Deploy complete! 2020-05-18T21:30:31.894 DEBUG:teuthology.parallel:result is None 2020-05-18T21:30:32.103 INFO:teuthology.lock.ops:Checking smithi196.front.sepia.ceph.com 2020-05-18T21:30:32.105 INFO:teuthology.lock.ops:New key found. Updating... 2020-05-18T21:30:32.137 INFO:teuthology.lock.ops:Checking smithi080.front.sepia.ceph.com 2020-05-18T21:30:32.138 INFO:teuthology.lock.ops:New key found. Updating... 2020-05-18T21:30:32.159 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: set os type and version on server 2020-05-18T21:30:32.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T21:30:32.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:32.207 INFO:teuthology.orchestra.run.smithi196:> true 2020-05-18T21:30:32.325 INFO:teuthology.orchestra.run.smithi196:> uname -m 2020-05-18T21:30:32.374 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2020-05-18T21:30:32.374 INFO:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2020-05-18T21:30:32.381 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="Ubuntu" 2020-05-18T21:30:32.382 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="20.04 LTS (Focal Fossa)" 2020-05-18T21:30:32.382 INFO:teuthology.orchestra.run.smithi196.stdout:ID=ubuntu 2020-05-18T21:30:32.382 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE=debian 2020-05-18T21:30:32.383 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="Ubuntu 20.04 LTS" 2020-05-18T21:30:32.383 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="20.04" 2020-05-18T21:30:32.383 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.ubuntu.com/" 2020-05-18T21:30:32.383 INFO:teuthology.orchestra.run.smithi196.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-05-18T21:30:32.384 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-05-18T21:30:32.384 INFO:teuthology.orchestra.run.smithi196.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-05-18T21:30:32.384 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_CODENAME=focal 2020-05-18T21:30:32.384 INFO:teuthology.orchestra.run.smithi196.stdout:UBUNTU_CODENAME=focal 2020-05-18T21:30:32.386 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2020-05-18T21:30:32.410 INFO:teuthology.lock.ops:Updating [smithi080.front.sepia.ceph.com]: set os type and version on server 2020-05-18T21:30:32.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T21:30:32.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:32.471 INFO:teuthology.orchestra.run.smithi080:> true 2020-05-18T21:30:32.591 INFO:teuthology.orchestra.run.smithi080:> uname -m 2020-05-18T21:30:32.598 INFO:teuthology.orchestra.run.smithi080.stdout:x86_64 2020-05-18T21:30:32.598 INFO:teuthology.orchestra.run.smithi080:> cat /etc/os-release 2020-05-18T21:30:32.646 INFO:teuthology.orchestra.run.smithi080.stdout:NAME="Ubuntu" 2020-05-18T21:30:32.647 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION="20.04 LTS (Focal Fossa)" 2020-05-18T21:30:32.647 INFO:teuthology.orchestra.run.smithi080.stdout:ID=ubuntu 2020-05-18T21:30:32.647 INFO:teuthology.orchestra.run.smithi080.stdout:ID_LIKE=debian 2020-05-18T21:30:32.648 INFO:teuthology.orchestra.run.smithi080.stdout:PRETTY_NAME="Ubuntu 20.04 LTS" 2020-05-18T21:30:32.648 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_ID="20.04" 2020-05-18T21:30:32.648 INFO:teuthology.orchestra.run.smithi080.stdout:HOME_URL="https://www.ubuntu.com/" 2020-05-18T21:30:32.648 INFO:teuthology.orchestra.run.smithi080.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-05-18T21:30:32.649 INFO:teuthology.orchestra.run.smithi080.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-05-18T21:30:32.649 INFO:teuthology.orchestra.run.smithi080.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-05-18T21:30:32.649 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_CODENAME=focal 2020-05-18T21:30:32.649 INFO:teuthology.orchestra.run.smithi080.stdout:UBUNTU_CODENAME=focal 2020-05-18T21:30:32.650 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server 2020-05-18T21:30:32.673 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2020-05-18T21:30:32.708 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi080.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXqMrU7EmXoNhLeHD+zrdPTLqVqhGYFJLcBsbpiCRPDWWApstvKWCD452dNgsr0V+7LJZpP+DYjaDciuXYLvJIyTfi+E3jtNqpr3ypIsM13lJhYJYqVlX8B6VbfQWjnEAjVnOv3fgCiU14uEGAW2gSBq9FqAKoDXOxHHRRuLJhMJBDijDVJX9+1JjgxX31Nz1vQfMtfWpQLa7ii4n8w/ByL65WSTWg47lhKDnuzbs3jabP02mTziz5g7yoNlg3j0F7jT01L2Lbj3iYF08n0q7BiRB7/PAtrksm4XkVDoYeF+/YQ/EZgYx9vp4tZ7Ewx6CBAVxbgNkRvGRi4wkuDqCEMMIlM6Yex56eLlZ41OXooGJ+fREfFsIlA8+gAX4GUr/OZJtOjxbm4+o+y3WQaafccL6Uo4L/wcvssHPRWEyHju2ZvXUQSfeXvD6N23otMvqvD672rUumEAThDfjqcXp/Mu6FIe5Q0WFisWFe5bGEdN6mg8YFpqNjb0wO1VzyiuE= smithi196.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXlKs/4Iub2TYjf6lRk+Xq/mWExXCpHbHiIDeEb2iyMCktl8JsJPbDS5GK1XVNP/tOrOCv+4Z8j3VigP/e17wirRvLgwXqdIU3neAxHgFVbjWomxnS74couN7s3gPpBtjC1Bp289v1Ofz8as1jjaj0lBdPUgR/l4N8alms7O9wR+wmWzrRbjiLAyOoaUcU0T91SqO6i/UEXNSgclq2vn9Gd2EX0QhoP6L+BP/xFSU3jAYRoLSI5Ls6uL3sP1PIF24nMP3EFoSFO8dvylVakjxSElH1O8EQcjBi7bd1dxfJdRFI93hwiDr66SarR8yLXHNX6/TvQjaQER/LLCmIedslZlHlc6vVyTCHhaY7+xmKfc2IdpdIKMVFyh9k2qpXi4N0edmOQziNpVMIbfsue0P9PmYcpNsrJ/qvWMGDPKtg8WwNPxUPvlQLXVc9Euj6mGH/K7uAkAy1gOjZOX86PzqtAXI2ITWF9QUJ/HeehvIuxip1Rp4Q7Hq59cSDGhzQ78c= 2020-05-18T21:30:32.709 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-05-18T21:30:32.764 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-05-18T21:30:32.773 INFO:teuthology.task.internal:Saving configuration 2020-05-18T21:30:32.794 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-05-18T21:30:32.807 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-05-18T21:30:32.824 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-05-18 21:25:09.403742', 'description': '/home/teuthworker/archive/pdonnell-2020-05-18_20:49:47-fs-wip-pdonnell-testing-20200516.030215-distro-basic-smithi/5067934', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIC9JlmVChcw+rw+NBVqTMpRMo1v1M37Cv7SKt3cK3151', 'os_version': '20.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi196.front.sepia.ceph.com'} 2020-05-18T21:30:32.843 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-05-18 21:25:09.403762', 'description': '/home/teuthworker/archive/pdonnell-2020-05-18_20:49:47-fs-wip-pdonnell-testing-20200516.030215-distro-basic-smithi/5067934', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMbZwubYIMTmjXioxEKXIIj7EE8SIjQwB78BphANxkiG', 'os_version': '20.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi080.front.sepia.ceph.com'} 2020-05-18T21:30:32.843 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-05-18T21:30:32.854 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2020-05-18T21:30:32.854 INFO:teuthology.task.internal:roles: ubuntu@smithi080.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2020-05-18T21:30:32.855 INFO:teuthology.run_tasks:Running task console_log... 2020-05-18T21:30:32.939 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa2740b1488>, signals=[15]) 2020-05-18T21:30:32.941 INFO:teuthology.run_tasks:Running task internal.connect... 2020-05-18T21:30:32.951 INFO:teuthology.task.internal:Opening connections... 2020-05-18T21:30:32.951 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2020-05-18T21:30:32.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:33.002 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi080.front.sepia.ceph.com 2020-05-18T21:30:33.005 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:30:33.113 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-05-18T21:30:33.126 INFO:teuthology.orchestra.run.smithi196:> uname -m 2020-05-18T21:30:33.133 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2020-05-18T21:30:33.134 INFO:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2020-05-18T21:30:33.188 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="Ubuntu" 2020-05-18T21:30:33.189 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="20.04 LTS (Focal Fossa)" 2020-05-18T21:30:33.189 INFO:teuthology.orchestra.run.smithi196.stdout:ID=ubuntu 2020-05-18T21:30:33.189 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE=debian 2020-05-18T21:30:33.189 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="Ubuntu 20.04 LTS" 2020-05-18T21:30:33.190 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="20.04" 2020-05-18T21:30:33.190 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.ubuntu.com/" 2020-05-18T21:30:33.190 INFO:teuthology.orchestra.run.smithi196.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-05-18T21:30:33.190 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-05-18T21:30:33.196 INFO:teuthology.orchestra.run.smithi196.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-05-18T21:30:33.197 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_CODENAME=focal 2020-05-18T21:30:33.197 INFO:teuthology.orchestra.run.smithi196.stdout:UBUNTU_CODENAME=focal 2020-05-18T21:30:33.198 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2020-05-18T21:30:33.224 INFO:teuthology.orchestra.run.smithi080:> uname -m 2020-05-18T21:30:33.232 INFO:teuthology.orchestra.run.smithi080.stdout:x86_64 2020-05-18T21:30:33.233 INFO:teuthology.orchestra.run.smithi080:> cat /etc/os-release 2020-05-18T21:30:33.282 INFO:teuthology.orchestra.run.smithi080.stdout:NAME="Ubuntu" 2020-05-18T21:30:33.283 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION="20.04 LTS (Focal Fossa)" 2020-05-18T21:30:33.283 INFO:teuthology.orchestra.run.smithi080.stdout:ID=ubuntu 2020-05-18T21:30:33.283 INFO:teuthology.orchestra.run.smithi080.stdout:ID_LIKE=debian 2020-05-18T21:30:33.284 INFO:teuthology.orchestra.run.smithi080.stdout:PRETTY_NAME="Ubuntu 20.04 LTS" 2020-05-18T21:30:33.284 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_ID="20.04" 2020-05-18T21:30:33.284 INFO:teuthology.orchestra.run.smithi080.stdout:HOME_URL="https://www.ubuntu.com/" 2020-05-18T21:30:33.284 INFO:teuthology.orchestra.run.smithi080.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-05-18T21:30:33.284 INFO:teuthology.orchestra.run.smithi080.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-05-18T21:30:33.285 INFO:teuthology.orchestra.run.smithi080.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-05-18T21:30:33.285 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_CODENAME=focal 2020-05-18T21:30:33.285 INFO:teuthology.orchestra.run.smithi080.stdout:UBUNTU_CODENAME=focal 2020-05-18T21:30:33.286 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server 2020-05-18T21:30:33.307 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-05-18T21:30:33.333 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-05-18T21:30:33.343 INFO:teuthology.task.internal:Checking for old test directory... 2020-05-18T21:30:33.344 INFO:teuthology.orchestra.run.smithi080:> test '!' -e /home/ubuntu/cephtest 2020-05-18T21:30:33.353 INFO:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2020-05-18T21:30:33.359 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-05-18T21:30:33.369 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-05-18T21:30:33.370 INFO:teuthology.orchestra.run.smithi080:> test -z $(ls -A /var/lib/ceph) 2020-05-18T21:30:33.402 INFO:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2020-05-18T21:30:33.416 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-05-18T21:30:33.461 INFO:teuthology.run_tasks:Running task kernel... 2020-05-18T21:30:33.502 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'mds.a': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mds.b': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}} 2020-05-18T21:30:33.504 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-05-18T21:30:33.504 INFO:teuthology.orchestra.run.smithi196:> uname -r 2020-05-18T21:30:33.512 INFO:teuthology.orchestra.run.smithi196.stdout:5.4.0-26-generic 2020-05-18T21:30:33.513 INFO:teuthology.task.kernel:Running kernel on smithi196: 5.4.0-26-generic 2020-05-18T21:30:33.513 INFO:teuthology.orchestra.run.smithi196:> sudo apt-get clean 2020-05-18T21:30:33.692 INFO:teuthology.orchestra.run.smithi196:> sudo apt-get update 2020-05-18T21:30:33.854 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [107 kB] 2020-05-18T21:30:33.965 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2020-05-18T21:30:34.046 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [107 kB] 2020-05-18T21:30:34.177 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [23.4 kB] 2020-05-18T21:30:34.211 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [61.2 kB] 2020-05-18T21:30:34.217 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [23.0 kB] 2020-05-18T21:30:34.220 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [1,592 B] 2020-05-18T21:30:34.221 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [4,180 B] 2020-05-18T21:30:34.221 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [1,360 B] 2020-05-18T21:30:34.222 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [8,428 B] 2020-05-18T21:30:34.222 INFO:teuthology.orchestra.run.smithi196.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [27.5 kB] 2020-05-18T21:30:34.226 INFO:teuthology.orchestra.run.smithi196.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [12.1 kB] 2020-05-18T21:30:34.227 INFO:teuthology.orchestra.run.smithi196.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [784 B] 2020-05-18T21:30:34.227 INFO:teuthology.orchestra.run.smithi196.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [1,172 B] 2020-05-18T21:30:34.237 INFO:teuthology.orchestra.run.smithi196.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [540 B] 2020-05-18T21:30:34.389 INFO:teuthology.orchestra.run.smithi196.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-backports InRelease [98.3 kB] 2020-05-18T21:30:34.527 INFO:teuthology.orchestra.run.smithi196.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [48.5 kB] 2020-05-18T21:30:34.548 INFO:teuthology.orchestra.run.smithi196.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [101 kB] 2020-05-18T21:30:34.595 INFO:teuthology.orchestra.run.smithi196.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [36.9 kB] 2020-05-18T21:30:34.613 INFO:teuthology.orchestra.run.smithi196.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [2,312 B] 2020-05-18T21:30:34.616 INFO:teuthology.orchestra.run.smithi196.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [4,180 B] 2020-05-18T21:30:34.618 INFO:teuthology.orchestra.run.smithi196.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [1,360 B] 2020-05-18T21:30:34.632 INFO:teuthology.orchestra.run.smithi196.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [48.6 kB] 2020-05-18T21:30:34.635 INFO:teuthology.orchestra.run.smithi196.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [22.1 kB] 2020-05-18T21:30:34.655 INFO:teuthology.orchestra.run.smithi196.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [20.4 kB] 2020-05-18T21:30:34.656 INFO:teuthology.orchestra.run.smithi196.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [1,380 B] 2020-05-18T21:30:34.657 INFO:teuthology.orchestra.run.smithi196.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [1,172 B] 2020-05-18T21:30:34.712 INFO:teuthology.orchestra.run.smithi196.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [540 B] 2020-05-18T21:30:34.742 INFO:teuthology.orchestra.run.smithi196.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [1,956 B] 2020-05-18T21:30:34.743 INFO:teuthology.orchestra.run.smithi196.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [2,792 B] 2020-05-18T21:30:34.743 INFO:teuthology.orchestra.run.smithi196.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [1,280 B] 2020-05-18T21:30:34.744 INFO:teuthology.orchestra.run.smithi196.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [188 B] 2020-05-18T21:30:36.124 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 772 kB in 1s (818 kB/s) 2020-05-18T21:30:37.069 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2020-05-18T21:30:37.088 INFO:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-05-18T21:30:37.153 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2020-05-18T21:30:37.287 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2020-05-18T21:30:37.288 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2020-05-18T21:30:37.360 INFO:teuthology.orchestra.run.smithi196.stderr:E: Unable to locate package linux-image-current-generic 2020-05-18T21:30:37.362 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-05-18T21:30:37.362 INFO:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-05-18T21:30:37.427 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2020-05-18T21:30:37.565 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2020-05-18T21:30:37.566 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2020-05-18T21:30:37.696 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2020-05-18T21:30:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: linux-generic linux-headers-5.4.0-29 linux-headers-5.4.0-29-generic 2020-05-18T21:30:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: linux-headers-generic linux-image-5.4.0-29-generic 2020-05-18T21:30:37.696 INFO:teuthology.orchestra.run.smithi196.stdout: linux-modules-5.4.0-29-generic linux-modules-extra-5.4.0-29-generic 2020-05-18T21:30:37.697 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2020-05-18T21:30:37.697 INFO:teuthology.orchestra.run.smithi196.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2020-05-18T21:30:37.698 INFO:teuthology.orchestra.run.smithi196.stdout:Recommended packages: 2020-05-18T21:30:37.698 INFO:teuthology.orchestra.run.smithi196.stdout: thermald 2020-05-18T21:30:37.725 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2020-05-18T21:30:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: linux-headers-5.4.0-29 linux-headers-5.4.0-29-generic 2020-05-18T21:30:37.725 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.4.0-29-generic linux-modules-5.4.0-29-generic 2020-05-18T21:30:37.726 INFO:teuthology.orchestra.run.smithi196.stdout: linux-modules-extra-5.4.0-29-generic 2020-05-18T21:30:37.726 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages will be upgraded: 2020-05-18T21:30:37.727 INFO:teuthology.orchestra.run.smithi196.stdout: linux-generic linux-headers-generic linux-image-generic 2020-05-18T21:30:37.905 INFO:teuthology.orchestra.run.smithi196.stdout:3 upgraded, 5 newly installed, 0 to remove and 50 not upgraded. 2020-05-18T21:30:37.906 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 73.9 MB of archives. 2020-05-18T21:30:37.906 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 359 MB of additional disk space will be used. 2020-05-18T21:30:37.906 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-29-generic amd64 5.4.0-29.33 [14.3 MB] 2020-05-18T21:30:39.023 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-29-generic amd64 5.4.0-29.33 [8,871 kB] 2020-05-18T21:30:39.270 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-29-generic amd64 5.4.0-29.33 [38.6 MB] 2020-05-18T21:30:40.516 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.29.34 [1,900 B] 2020-05-18T21:30:40.517 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.29.34 [2,712 B] 2020-05-18T21:30:40.517 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-29 all 5.4.0-29.33 [10.9 MB] 2020-05-18T21:30:41.082 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-29-generic amd64 5.4.0-29.33 [1,214 kB] 2020-05-18T21:30:41.105 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.29.34 [2,592 B] 2020-05-18T21:30:41.577 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 73.9 MB in 3s (21.9 MB/s) 2020-05-18T21:30:41.770 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-modules-5.4.0-29-generic. 2020-05-18T21:30:43.547 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 113167 files and directories currently installed.) 2020-05-18T21:30:43.553 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../0-linux-modules-5.4.0-29-generic_5.4.0-29.33_amd64.deb ... 2020-05-18T21:30:43.649 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-modules-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:30:45.374 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-image-5.4.0-29-generic. 2020-05-18T21:30:45.391 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../1-linux-image-5.4.0-29-generic_5.4.0-29.33_amd64.deb ... 2020-05-18T21:30:45.490 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-image-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:30:46.460 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-29-generic. 2020-05-18T21:30:46.476 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-29-generic_5.4.0-29.33_amd64.deb ... 2020-05-18T21:30:46.518 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-modules-extra-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:30:51.217 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../3-linux-generic_5.4.0.29.34_amd64.deb ... 2020-05-18T21:30:51.332 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-generic (5.4.0.29.34) over (5.4.0.26.32) ... 2020-05-18T21:30:51.674 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.29.34_amd64.deb ... 2020-05-18T21:30:51.802 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-image-generic (5.4.0.29.34) over (5.4.0.26.32) ... 2020-05-18T21:30:52.130 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-headers-5.4.0-29. 2020-05-18T21:30:52.142 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../5-linux-headers-5.4.0-29_5.4.0-29.33_all.deb ... 2020-05-18T21:30:52.180 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-headers-5.4.0-29 (5.4.0-29.33) ... 2020-05-18T21:30:57.961 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-headers-5.4.0-29-generic. 2020-05-18T21:30:57.984 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../6-linux-headers-5.4.0-29-generic_5.4.0-29.33_amd64.deb ... 2020-05-18T21:30:58.027 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-headers-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:31:00.538 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.29.34_amd64.deb ... 2020-05-18T21:31:00.664 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-headers-generic (5.4.0.29.34) over (5.4.0.26.32) ... 2020-05-18T21:31:01.004 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-modules-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:31:02.145 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-image-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:31:03.253 INFO:teuthology.orchestra.run.smithi196.stdout:I: /boot/vmlinuz is now a symlink to vmlinuz-5.4.0-29-generic 2020-05-18T21:31:03.253 INFO:teuthology.orchestra.run.smithi196.stdout:I: /boot/initrd.img is now a symlink to initrd.img-5.4.0-29-generic 2020-05-18T21:31:03.380 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-headers-5.4.0-29 (5.4.0-29.33) ... 2020-05-18T21:31:03.506 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-modules-extra-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:31:04.608 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-image-generic (5.4.0.29.34) ... 2020-05-18T21:31:04.734 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-headers-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:31:04.860 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-headers-generic (5.4.0.29.34) ... 2020-05-18T21:31:05.003 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-generic (5.4.0.29.34) ... 2020-05-18T21:31:05.129 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for linux-image-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:31:05.323 INFO:teuthology.orchestra.run.smithi196.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-05-18T21:31:05.324 INFO:teuthology.orchestra.run.smithi196.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-29-generic 2020-05-18T21:31:26.796 INFO:teuthology.orchestra.run.smithi196.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-05-18T21:31:26.797 INFO:teuthology.orchestra.run.smithi196.stdout:Sourcing file `/etc/default/grub' 2020-05-18T21:31:26.812 INFO:teuthology.orchestra.run.smithi196.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2020-05-18T21:31:26.828 INFO:teuthology.orchestra.run.smithi196.stdout:Generating grub configuration file ... 2020-05-18T21:31:27.151 INFO:teuthology.orchestra.run.smithi196.stdout:Found linux image: /boot/vmlinuz-5.4.0-29-generic 2020-05-18T21:31:27.187 INFO:teuthology.orchestra.run.smithi196.stdout:Found initrd image: /boot/initrd.img-5.4.0-29-generic 2020-05-18T21:31:27.588 INFO:teuthology.orchestra.run.smithi196.stdout:Found linux image: /boot/vmlinuz-5.4.0-26-generic 2020-05-18T21:31:27.599 INFO:teuthology.orchestra.run.smithi196.stdout:Found initrd image: /boot/initrd.img-5.4.0-26-generic 2020-05-18T21:31:30.183 INFO:teuthology.orchestra.run.smithi196.stdout:done 2020-05-18T21:31:30.522 INFO:teuthology.orchestra.run.smithi196:> dpkg -s linux-image-generic 2020-05-18T21:31:30.553 INFO:teuthology.orchestra.run.smithi196.stdout:Package: linux-image-generic 2020-05-18T21:31:30.554 INFO:teuthology.orchestra.run.smithi196.stdout:Status: install ok installed 2020-05-18T21:31:30.554 INFO:teuthology.orchestra.run.smithi196.stdout:Priority: optional 2020-05-18T21:31:30.554 INFO:teuthology.orchestra.run.smithi196.stdout:Section: kernel 2020-05-18T21:31:30.554 INFO:teuthology.orchestra.run.smithi196.stdout:Installed-Size: 17 2020-05-18T21:31:30.555 INFO:teuthology.orchestra.run.smithi196.stdout:Maintainer: Ubuntu Kernel Team 2020-05-18T21:31:30.555 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: amd64 2020-05-18T21:31:30.555 INFO:teuthology.orchestra.run.smithi196.stdout:Source: linux-meta 2020-05-18T21:31:30.555 INFO:teuthology.orchestra.run.smithi196.stdout:Version: 5.4.0.29.34 2020-05-18T21:31:30.555 INFO:teuthology.orchestra.run.smithi196.stdout:Provides: virtualbox-guest-modules (= 6.1.6-dfsg-1), wireguard-modules (= 1.0.20200413-1), zfs-modules (= 0.8.3-1ubuntu12) 2020-05-18T21:31:30.556 INFO:teuthology.orchestra.run.smithi196.stdout:Depends: linux-image-5.4.0-29-generic, linux-modules-extra-5.4.0-29-generic, linux-firmware, intel-microcode, amd64-microcode 2020-05-18T21:31:30.556 INFO:teuthology.orchestra.run.smithi196.stdout:Recommends: thermald 2020-05-18T21:31:30.556 INFO:teuthology.orchestra.run.smithi196.stdout:Description: Generic Linux kernel image 2020-05-18T21:31:30.556 INFO:teuthology.orchestra.run.smithi196.stdout: This package will always depend on the latest generic kernel image 2020-05-18T21:31:30.557 INFO:teuthology.orchestra.run.smithi196.stdout: available. 2020-05-18T21:31:30.558 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-26-generic Expected: 5.4.0-29-generic 2020-05-18T21:31:30.558 INFO:teuthology.orchestra.run.smithi080:> uname -r 2020-05-18T21:31:30.565 INFO:teuthology.orchestra.run.smithi080.stdout:5.4.0-26-generic 2020-05-18T21:31:30.565 INFO:teuthology.task.kernel:Running kernel on smithi080: 5.4.0-26-generic 2020-05-18T21:31:30.566 INFO:teuthology.orchestra.run.smithi080:> sudo apt-get clean 2020-05-18T21:31:30.695 INFO:teuthology.orchestra.run.smithi080:> sudo apt-get update 2020-05-18T21:31:30.974 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2020-05-18T21:31:30.974 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [107 kB] 2020-05-18T21:31:31.039 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [107 kB] 2020-05-18T21:31:31.376 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [98.3 kB] 2020-05-18T21:31:31.443 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [61.2 kB] 2020-05-18T21:31:31.566 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [23.4 kB] 2020-05-18T21:31:31.576 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [48.5 kB] 2020-05-18T21:31:31.578 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [23.0 kB] 2020-05-18T21:31:31.590 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [1,592 B] 2020-05-18T21:31:31.590 INFO:teuthology.orchestra.run.smithi080.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [4,180 B] 2020-05-18T21:31:31.592 INFO:teuthology.orchestra.run.smithi080.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [1,360 B] 2020-05-18T21:31:31.593 INFO:teuthology.orchestra.run.smithi080.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [27.5 kB] 2020-05-18T21:31:31.605 INFO:teuthology.orchestra.run.smithi080.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [8,428 B] 2020-05-18T21:31:31.610 INFO:teuthology.orchestra.run.smithi080.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [12.1 kB] 2020-05-18T21:31:31.620 INFO:teuthology.orchestra.run.smithi080.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [784 B] 2020-05-18T21:31:31.620 INFO:teuthology.orchestra.run.smithi080.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [1,172 B] 2020-05-18T21:31:31.659 INFO:teuthology.orchestra.run.smithi080.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [540 B] 2020-05-18T21:31:31.680 INFO:teuthology.orchestra.run.smithi080.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [101 kB] 2020-05-18T21:31:31.725 INFO:teuthology.orchestra.run.smithi080.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [36.9 kB] 2020-05-18T21:31:31.741 INFO:teuthology.orchestra.run.smithi080.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [2,312 B] 2020-05-18T21:31:31.742 INFO:teuthology.orchestra.run.smithi080.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [4,180 B] 2020-05-18T21:31:31.744 INFO:teuthology.orchestra.run.smithi080.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [1,360 B] 2020-05-18T21:31:31.745 INFO:teuthology.orchestra.run.smithi080.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [48.6 kB] 2020-05-18T21:31:31.766 INFO:teuthology.orchestra.run.smithi080.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [22.1 kB] 2020-05-18T21:31:31.776 INFO:teuthology.orchestra.run.smithi080.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [20.4 kB] 2020-05-18T21:31:31.785 INFO:teuthology.orchestra.run.smithi080.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [1,380 B] 2020-05-18T21:31:31.786 INFO:teuthology.orchestra.run.smithi080.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [1,172 B] 2020-05-18T21:31:31.806 INFO:teuthology.orchestra.run.smithi080.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [540 B] 2020-05-18T21:31:31.887 INFO:teuthology.orchestra.run.smithi080.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [1,956 B] 2020-05-18T21:31:31.888 INFO:teuthology.orchestra.run.smithi080.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [2,792 B] 2020-05-18T21:31:31.888 INFO:teuthology.orchestra.run.smithi080.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [1,280 B] 2020-05-18T21:31:31.888 INFO:teuthology.orchestra.run.smithi080.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [188 B] 2020-05-18T21:31:33.459 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 772 kB in 1s (690 kB/s) 2020-05-18T21:31:34.446 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2020-05-18T21:31:34.464 INFO:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-05-18T21:31:34.529 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2020-05-18T21:31:34.662 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2020-05-18T21:31:34.663 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2020-05-18T21:31:34.737 INFO:teuthology.orchestra.run.smithi080.stderr:E: Unable to locate package linux-image-current-generic 2020-05-18T21:31:34.738 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-05-18T21:31:34.739 INFO:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-05-18T21:31:34.804 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2020-05-18T21:31:34.938 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2020-05-18T21:31:34.938 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2020-05-18T21:31:35.069 INFO:teuthology.orchestra.run.smithi080.stdout:The following additional packages will be installed: 2020-05-18T21:31:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: linux-generic linux-headers-5.4.0-29 linux-headers-5.4.0-29-generic 2020-05-18T21:31:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: linux-headers-generic linux-image-5.4.0-29-generic 2020-05-18T21:31:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: linux-modules-5.4.0-29-generic linux-modules-extra-5.4.0-29-generic 2020-05-18T21:31:35.070 INFO:teuthology.orchestra.run.smithi080.stdout:Suggested packages: 2020-05-18T21:31:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2020-05-18T21:31:35.071 INFO:teuthology.orchestra.run.smithi080.stdout:Recommended packages: 2020-05-18T21:31:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: thermald 2020-05-18T21:31:35.100 INFO:teuthology.orchestra.run.smithi080.stdout:The following NEW packages will be installed: 2020-05-18T21:31:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: linux-headers-5.4.0-29 linux-headers-5.4.0-29-generic 2020-05-18T21:31:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: linux-image-5.4.0-29-generic linux-modules-5.4.0-29-generic 2020-05-18T21:31:35.102 INFO:teuthology.orchestra.run.smithi080.stdout: linux-modules-extra-5.4.0-29-generic 2020-05-18T21:31:35.102 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages will be upgraded: 2020-05-18T21:31:35.102 INFO:teuthology.orchestra.run.smithi080.stdout: linux-generic linux-headers-generic linux-image-generic 2020-05-18T21:31:35.282 INFO:teuthology.orchestra.run.smithi080.stdout:3 upgraded, 5 newly installed, 0 to remove and 50 not upgraded. 2020-05-18T21:31:35.282 INFO:teuthology.orchestra.run.smithi080.stdout:Need to get 73.9 MB of archives. 2020-05-18T21:31:35.282 INFO:teuthology.orchestra.run.smithi080.stdout:After this operation, 359 MB of additional disk space will be used. 2020-05-18T21:31:35.282 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-29-generic amd64 5.4.0-29.33 [14.3 MB] 2020-05-18T21:31:36.398 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-29-generic amd64 5.4.0-29.33 [8,871 kB] 2020-05-18T21:31:36.592 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-29-generic amd64 5.4.0-29.33 [38.6 MB] 2020-05-18T21:31:37.800 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.29.34 [1,900 B] 2020-05-18T21:31:37.801 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.29.34 [2,712 B] 2020-05-18T21:31:37.801 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-29 all 5.4.0-29.33 [10.9 MB] 2020-05-18T21:31:38.123 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-29-generic amd64 5.4.0-29.33 [1,214 kB] 2020-05-18T21:31:38.157 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.29.34 [2,592 B] 2020-05-18T21:31:38.608 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 73.9 MB in 3s (24.2 MB/s) 2020-05-18T21:31:38.863 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-modules-5.4.0-29-generic. 2020-05-18T21:31:40.719 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 113167 files and directories currently installed.) 2020-05-18T21:31:40.724 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../0-linux-modules-5.4.0-29-generic_5.4.0-29.33_amd64.deb ... 2020-05-18T21:31:40.836 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-modules-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:31:42.762 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-image-5.4.0-29-generic. 2020-05-18T21:31:42.777 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../1-linux-image-5.4.0-29-generic_5.4.0-29.33_amd64.deb ... 2020-05-18T21:31:42.864 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-image-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:31:43.882 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-29-generic. 2020-05-18T21:31:43.896 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-29-generic_5.4.0-29.33_amd64.deb ... 2020-05-18T21:31:43.939 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-modules-extra-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:31:48.515 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../3-linux-generic_5.4.0.29.34_amd64.deb ... 2020-05-18T21:31:48.617 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-generic (5.4.0.29.34) over (5.4.0.26.32) ... 2020-05-18T21:31:48.920 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.29.34_amd64.deb ... 2020-05-18T21:31:49.071 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-image-generic (5.4.0.29.34) over (5.4.0.26.32) ... 2020-05-18T21:31:49.390 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-headers-5.4.0-29. 2020-05-18T21:31:49.405 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../5-linux-headers-5.4.0-29_5.4.0-29.33_all.deb ... 2020-05-18T21:31:49.440 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-headers-5.4.0-29 (5.4.0-29.33) ... 2020-05-18T21:31:54.566 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-headers-5.4.0-29-generic. 2020-05-18T21:31:54.588 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../6-linux-headers-5.4.0-29-generic_5.4.0-29.33_amd64.deb ... 2020-05-18T21:31:54.632 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-headers-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:31:57.184 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.29.34_amd64.deb ... 2020-05-18T21:31:57.306 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-headers-generic (5.4.0.29.34) over (5.4.0.26.32) ... 2020-05-18T21:31:57.641 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-modules-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:31:58.754 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-image-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:31:59.823 INFO:teuthology.orchestra.run.smithi080.stdout:I: /boot/vmlinuz is now a symlink to vmlinuz-5.4.0-29-generic 2020-05-18T21:31:59.823 INFO:teuthology.orchestra.run.smithi080.stdout:I: /boot/initrd.img is now a symlink to initrd.img-5.4.0-29-generic 2020-05-18T21:31:59.956 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-headers-5.4.0-29 (5.4.0-29.33) ... 2020-05-18T21:32:00.074 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-modules-extra-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:32:01.192 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-image-generic (5.4.0.29.34) ... 2020-05-18T21:32:01.293 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-headers-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:32:01.410 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-headers-generic (5.4.0.29.34) ... 2020-05-18T21:32:01.536 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-generic (5.4.0.29.34) ... 2020-05-18T21:32:01.663 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for linux-image-5.4.0-29-generic (5.4.0-29.33) ... 2020-05-18T21:32:01.836 INFO:teuthology.orchestra.run.smithi080.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-05-18T21:32:01.836 INFO:teuthology.orchestra.run.smithi080.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-29-generic 2020-05-18T21:32:23.543 INFO:teuthology.orchestra.run.smithi080.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-05-18T21:32:23.544 INFO:teuthology.orchestra.run.smithi080.stdout:Sourcing file `/etc/default/grub' 2020-05-18T21:32:23.567 INFO:teuthology.orchestra.run.smithi080.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2020-05-18T21:32:23.580 INFO:teuthology.orchestra.run.smithi080.stdout:Generating grub configuration file ... 2020-05-18T21:32:23.863 INFO:teuthology.orchestra.run.smithi080.stdout:Found linux image: /boot/vmlinuz-5.4.0-29-generic 2020-05-18T21:32:23.886 INFO:teuthology.orchestra.run.smithi080.stdout:Found initrd image: /boot/initrd.img-5.4.0-29-generic 2020-05-18T21:32:24.264 INFO:teuthology.orchestra.run.smithi080.stdout:Found linux image: /boot/vmlinuz-5.4.0-26-generic 2020-05-18T21:32:24.275 INFO:teuthology.orchestra.run.smithi080.stdout:Found initrd image: /boot/initrd.img-5.4.0-26-generic 2020-05-18T21:32:26.968 INFO:teuthology.orchestra.run.smithi080.stdout:done 2020-05-18T21:32:27.273 INFO:teuthology.orchestra.run.smithi080:> dpkg -s linux-image-generic 2020-05-18T21:32:27.301 INFO:teuthology.orchestra.run.smithi080.stdout:Package: linux-image-generic 2020-05-18T21:32:27.301 INFO:teuthology.orchestra.run.smithi080.stdout:Status: install ok installed 2020-05-18T21:32:27.301 INFO:teuthology.orchestra.run.smithi080.stdout:Priority: optional 2020-05-18T21:32:27.302 INFO:teuthology.orchestra.run.smithi080.stdout:Section: kernel 2020-05-18T21:32:27.302 INFO:teuthology.orchestra.run.smithi080.stdout:Installed-Size: 17 2020-05-18T21:32:27.302 INFO:teuthology.orchestra.run.smithi080.stdout:Maintainer: Ubuntu Kernel Team 2020-05-18T21:32:27.302 INFO:teuthology.orchestra.run.smithi080.stdout:Architecture: amd64 2020-05-18T21:32:27.302 INFO:teuthology.orchestra.run.smithi080.stdout:Source: linux-meta 2020-05-18T21:32:27.303 INFO:teuthology.orchestra.run.smithi080.stdout:Version: 5.4.0.29.34 2020-05-18T21:32:27.303 INFO:teuthology.orchestra.run.smithi080.stdout:Provides: virtualbox-guest-modules (= 6.1.6-dfsg-1), wireguard-modules (= 1.0.20200413-1), zfs-modules (= 0.8.3-1ubuntu12) 2020-05-18T21:32:27.303 INFO:teuthology.orchestra.run.smithi080.stdout:Depends: linux-image-5.4.0-29-generic, linux-modules-extra-5.4.0-29-generic, linux-firmware, intel-microcode, amd64-microcode 2020-05-18T21:32:27.303 INFO:teuthology.orchestra.run.smithi080.stdout:Recommends: thermald 2020-05-18T21:32:27.303 INFO:teuthology.orchestra.run.smithi080.stdout:Description: Generic Linux kernel image 2020-05-18T21:32:27.303 INFO:teuthology.orchestra.run.smithi080.stdout: This package will always depend on the latest generic kernel image 2020-05-18T21:32:27.304 INFO:teuthology.orchestra.run.smithi080.stdout: available. 2020-05-18T21:32:27.305 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-26-generic Expected: 5.4.0-29-generic 2020-05-18T21:32:27.305 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2020-05-18T21:32:27.305 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-05-18T21:32:27.306 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-05-18T21:32:27.306 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2020-05-18T21:32:27.306 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=None, version=distro) 2020-05-18T21:32:27.307 INFO:teuthology.orchestra.run.smithi196:> sudo apt-get clean 2020-05-18T21:32:27.429 INFO:teuthology.orchestra.run.smithi196:> sudo apt-get update 2020-05-18T21:32:27.713 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2020-05-18T21:32:27.714 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2020-05-18T21:32:27.782 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2020-05-18T21:32:27.866 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2020-05-18T21:32:29.036 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2020-05-18T21:32:29.054 INFO:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-05-18T21:32:29.118 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2020-05-18T21:32:29.251 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2020-05-18T21:32:29.252 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2020-05-18T21:32:29.326 INFO:teuthology.orchestra.run.smithi196.stderr:E: Unable to locate package linux-image-current-generic 2020-05-18T21:32:29.328 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-05-18T21:32:29.329 INFO:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-05-18T21:32:29.397 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2020-05-18T21:32:29.531 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2020-05-18T21:32:29.532 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2020-05-18T21:32:29.681 INFO:teuthology.orchestra.run.smithi196.stdout:linux-image-generic is already the newest version (5.4.0.29.34). 2020-05-18T21:32:29.681 INFO:teuthology.orchestra.run.smithi196.stdout:0 upgraded, 0 newly installed, 0 to remove and 50 not upgraded. 2020-05-18T21:32:29.683 INFO:teuthology.orchestra.run.smithi196:> dpkg -s linux-image-generic 2020-05-18T21:32:29.705 INFO:teuthology.orchestra.run.smithi196.stdout:Package: linux-image-generic 2020-05-18T21:32:29.705 INFO:teuthology.orchestra.run.smithi196.stdout:Status: install ok installed 2020-05-18T21:32:29.705 INFO:teuthology.orchestra.run.smithi196.stdout:Priority: optional 2020-05-18T21:32:29.706 INFO:teuthology.orchestra.run.smithi196.stdout:Section: kernel 2020-05-18T21:32:29.706 INFO:teuthology.orchestra.run.smithi196.stdout:Installed-Size: 17 2020-05-18T21:32:29.706 INFO:teuthology.orchestra.run.smithi196.stdout:Maintainer: Ubuntu Kernel Team 2020-05-18T21:32:29.706 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: amd64 2020-05-18T21:32:29.706 INFO:teuthology.orchestra.run.smithi196.stdout:Source: linux-meta 2020-05-18T21:32:29.707 INFO:teuthology.orchestra.run.smithi196.stdout:Version: 5.4.0.29.34 2020-05-18T21:32:29.707 INFO:teuthology.orchestra.run.smithi196.stdout:Provides: virtualbox-guest-modules (= 6.1.6-dfsg-1), wireguard-modules (= 1.0.20200413-1), zfs-modules (= 0.8.3-1ubuntu12) 2020-05-18T21:32:29.707 INFO:teuthology.orchestra.run.smithi196.stdout:Depends: linux-image-5.4.0-29-generic, linux-modules-extra-5.4.0-29-generic, linux-firmware, intel-microcode, amd64-microcode 2020-05-18T21:32:29.707 INFO:teuthology.orchestra.run.smithi196.stdout:Recommends: thermald 2020-05-18T21:32:29.707 INFO:teuthology.orchestra.run.smithi196.stdout:Description: Generic Linux kernel image 2020-05-18T21:32:29.707 INFO:teuthology.orchestra.run.smithi196.stdout: This package will always depend on the latest generic kernel image 2020-05-18T21:32:29.708 INFO:teuthology.orchestra.run.smithi196.stdout: available. 2020-05-18T21:32:29.709 INFO:teuthology.orchestra.run.smithi196:> mktemp 2020-05-18T21:32:29.713 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.OmdHq5UG6y 2020-05-18T21:32:29.714 INFO:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub/grub.cfg /tmp/tmp.OmdHq5UG6y 2020-05-18T21:32:29.793 INFO:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.OmdHq5UG6y 2020-05-18T21:32:29.830 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.OmdHq5UG6y is 8KB 2020-05-18T21:32:29.843 INFO:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.OmdHq5UG6y 2020-05-18T21:32:29.849 INFO:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-05-18T21:32:29.906 INFO:teuthology.orchestra.run.smithi196:> sudo sh -c 'cat > /etc/grub.d/01_ceph_kernel' && sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-05-18T21:32:29.942 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-29-generic 2020-05-18T21:32:29.942 INFO:teuthology.orchestra.run.smithi196:> sudo update-grub 2020-05-18T21:32:30.947 INFO:teuthology.orchestra.run.smithi196.stderr:Sourcing file `/etc/default/grub' 2020-05-18T21:32:30.953 INFO:teuthology.orchestra.run.smithi196.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2020-05-18T21:32:30.966 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2020-05-18T21:32:31.203 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.4.0-29-generic 2020-05-18T21:32:31.215 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.4.0-29-generic 2020-05-18T21:32:31.601 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.4.0-26-generic 2020-05-18T21:32:31.613 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.4.0-26-generic 2020-05-18T21:32:33.195 INFO:teuthology.orchestra.run.smithi196.stderr:done 2020-05-18T21:32:33.197 INFO:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2020-05-18T21:32:33.199 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2020-05-18T21:32:33.199 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi080.front.sepia.ceph.com, path=None, version=distro) 2020-05-18T21:32:33.200 INFO:teuthology.orchestra.run.smithi080:> sudo apt-get clean 2020-05-18T21:32:33.275 INFO:teuthology.orchestra.run.smithi080:> sudo apt-get update 2020-05-18T21:32:33.550 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2020-05-18T21:32:33.551 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2020-05-18T21:32:33.619 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2020-05-18T21:32:33.704 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2020-05-18T21:32:34.852 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2020-05-18T21:32:34.872 INFO:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-05-18T21:32:34.932 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2020-05-18T21:32:35.050 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2020-05-18T21:32:35.051 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2020-05-18T21:32:35.114 INFO:teuthology.orchestra.run.smithi080.stderr:E: Unable to locate package linux-image-current-generic 2020-05-18T21:32:35.116 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-05-18T21:32:35.117 INFO:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-05-18T21:32:35.182 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2020-05-18T21:32:35.310 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2020-05-18T21:32:35.310 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2020-05-18T21:32:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:linux-image-generic is already the newest version (5.4.0.29.34). 2020-05-18T21:32:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:0 upgraded, 0 newly installed, 0 to remove and 50 not upgraded. 2020-05-18T21:32:35.449 INFO:teuthology.orchestra.run.smithi080:> dpkg -s linux-image-generic 2020-05-18T21:32:35.471 INFO:teuthology.orchestra.run.smithi080.stdout:Package: linux-image-generic 2020-05-18T21:32:35.472 INFO:teuthology.orchestra.run.smithi080.stdout:Status: install ok installed 2020-05-18T21:32:35.472 INFO:teuthology.orchestra.run.smithi080.stdout:Priority: optional 2020-05-18T21:32:35.472 INFO:teuthology.orchestra.run.smithi080.stdout:Section: kernel 2020-05-18T21:32:35.472 INFO:teuthology.orchestra.run.smithi080.stdout:Installed-Size: 17 2020-05-18T21:32:35.472 INFO:teuthology.orchestra.run.smithi080.stdout:Maintainer: Ubuntu Kernel Team 2020-05-18T21:32:35.473 INFO:teuthology.orchestra.run.smithi080.stdout:Architecture: amd64 2020-05-18T21:32:35.473 INFO:teuthology.orchestra.run.smithi080.stdout:Source: linux-meta 2020-05-18T21:32:35.473 INFO:teuthology.orchestra.run.smithi080.stdout:Version: 5.4.0.29.34 2020-05-18T21:32:35.473 INFO:teuthology.orchestra.run.smithi080.stdout:Provides: virtualbox-guest-modules (= 6.1.6-dfsg-1), wireguard-modules (= 1.0.20200413-1), zfs-modules (= 0.8.3-1ubuntu12) 2020-05-18T21:32:35.473 INFO:teuthology.orchestra.run.smithi080.stdout:Depends: linux-image-5.4.0-29-generic, linux-modules-extra-5.4.0-29-generic, linux-firmware, intel-microcode, amd64-microcode 2020-05-18T21:32:35.474 INFO:teuthology.orchestra.run.smithi080.stdout:Recommends: thermald 2020-05-18T21:32:35.474 INFO:teuthology.orchestra.run.smithi080.stdout:Description: Generic Linux kernel image 2020-05-18T21:32:35.474 INFO:teuthology.orchestra.run.smithi080.stdout: This package will always depend on the latest generic kernel image 2020-05-18T21:32:35.474 INFO:teuthology.orchestra.run.smithi080.stdout: available. 2020-05-18T21:32:35.475 INFO:teuthology.orchestra.run.smithi080:> mktemp 2020-05-18T21:32:35.479 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.3Lj8d8qzKL 2020-05-18T21:32:35.480 INFO:teuthology.orchestra.run.smithi080:> sudo cp /boot/grub/grub.cfg /tmp/tmp.3Lj8d8qzKL 2020-05-18T21:32:35.537 INFO:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.3Lj8d8qzKL 2020-05-18T21:32:35.575 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.3Lj8d8qzKL is 8KB 2020-05-18T21:32:35.588 INFO:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.3Lj8d8qzKL 2020-05-18T21:32:35.594 INFO:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-05-18T21:32:35.648 INFO:teuthology.orchestra.run.smithi080:> sudo sh -c 'cat > /etc/grub.d/01_ceph_kernel' && sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-05-18T21:32:35.674 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-29-generic 2020-05-18T21:32:35.674 INFO:teuthology.orchestra.run.smithi080:> sudo update-grub 2020-05-18T21:32:36.839 INFO:teuthology.orchestra.run.smithi080.stderr:Sourcing file `/etc/default/grub' 2020-05-18T21:32:36.846 INFO:teuthology.orchestra.run.smithi080.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2020-05-18T21:32:36.860 INFO:teuthology.orchestra.run.smithi080.stderr:Generating grub configuration file ... 2020-05-18T21:32:37.103 INFO:teuthology.orchestra.run.smithi080.stderr:Found linux image: /boot/vmlinuz-5.4.0-29-generic 2020-05-18T21:32:37.115 INFO:teuthology.orchestra.run.smithi080.stderr:Found initrd image: /boot/initrd.img-5.4.0-29-generic 2020-05-18T21:32:37.429 INFO:teuthology.orchestra.run.smithi080.stderr:Found linux image: /boot/vmlinuz-5.4.0-26-generic 2020-05-18T21:32:37.441 INFO:teuthology.orchestra.run.smithi080.stderr:Found initrd image: /boot/initrd.img-5.4.0-26-generic 2020-05-18T21:32:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:done 2020-05-18T21:32:39.050 INFO:teuthology.orchestra.run.smithi080:> sudo shutdown -r now 2020-05-18T21:33:09.082 INFO:teuthology.misc:Re-opening connections... 2020-05-18T21:33:09.083 INFO:teuthology.misc:trying to connect to ubuntu@smithi196.front.sepia.ceph.com 2020-05-18T21:33:09.084 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T21:33:09.085 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:33:27.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2020-05-18T21:33:27.649 INFO:teuthology.misc:trying to connect to ubuntu@smithi080.front.sepia.ceph.com 2020-05-18T21:33:27.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T21:33:27.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:34:27.653 DEBUG:teuthology.orchestra.remote:timed out 2020-05-18T21:34:27.656 DEBUG:teuthology.misc:waited 78.57305526733398 2020-05-18T21:34:28.657 INFO:teuthology.misc:trying to connect to ubuntu@smithi196.front.sepia.ceph.com 2020-05-18T21:34:28.658 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T21:34:28.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:34:28.758 INFO:teuthology.orchestra.run.smithi196:> true 2020-05-18T21:34:29.141 DEBUG:teuthology.misc:waited 80.05753564834595 2020-05-18T21:34:30.142 INFO:teuthology.misc:trying to connect to ubuntu@smithi080.front.sepia.ceph.com 2020-05-18T21:34:30.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T21:34:30.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:34:30.346 INFO:teuthology.orchestra.run.smithi080:> true 2020-05-18T21:34:30.778 DEBUG:teuthology.misc:waited 81.69526219367981 2020-05-18T21:34:31.780 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2020-05-18T21:34:31.780 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-29-generic"... 2020-05-18T21:34:31.781 INFO:teuthology.orchestra.run.smithi196:> uname -r 2020-05-18T21:34:31.788 INFO:teuthology.orchestra.run.smithi196.stdout:5.4.0-29-generic 2020-05-18T21:34:31.789 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-29-generic vs 5.4.0-29-generic 2020-05-18T21:34:31.789 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-05-18T21:34:31.789 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2020-05-18T21:34:31.789 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-29-generic"... 2020-05-18T21:34:31.791 INFO:teuthology.orchestra.run.smithi080:> uname -r 2020-05-18T21:34:31.798 INFO:teuthology.orchestra.run.smithi080.stdout:5.4.0-29-generic 2020-05-18T21:34:31.798 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-29-generic vs 5.4.0-29-generic 2020-05-18T21:34:31.798 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-05-18T21:34:32.799 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-05-18T21:34:32.799 INFO:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-05-18T21:34:32.873 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2020-05-18T21:34:32.907 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2020-05-18T21:34:32.907 INFO:teuthology.orchestra.run.smithi080:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-05-18T21:34:33.021 INFO:teuthology.orchestra.run.smithi080.stdout:ttyS1 2020-05-18T21:34:33.044 INFO:teuthology.run_tasks:Running task internal.base... 2020-05-18T21:34:33.055 INFO:teuthology.task.internal:Creating test directory... 2020-05-18T21:34:33.056 INFO:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-05-18T21:34:33.059 INFO:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-05-18T21:34:33.080 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-05-18T21:34:33.091 INFO:teuthology.run_tasks:Running task internal.archive... 2020-05-18T21:34:33.104 INFO:teuthology.task.internal:Creating archive directory... 2020-05-18T21:34:33.105 INFO:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-05-18T21:34:33.107 INFO:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-05-18T21:34:33.134 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-05-18T21:34:33.145 INFO:teuthology.task.internal:Enabling coredump saving... 2020-05-18T21:34:33.145 INFO:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-05-18T21:34:33.164 INFO:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-05-18T21:34:33.189 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-05-18T21:34:33.200 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-05-18T21:34:33.202 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-05-18T21:34:33.212 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-05-18T21:34:33.214 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-05-18T21:34:33.225 INFO:teuthology.task.internal:Configuring sudo... 2020-05-18T21:34:33.226 INFO:teuthology.orchestra.run.smithi080:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-05-18T21:34:33.228 INFO:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-05-18T21:34:33.248 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-05-18T21:34:33.264 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-05-18T21:34:33.264 INFO:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-05-18T21:34:33.267 INFO:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-05-18T21:34:33.294 INFO:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-05-18T21:34:33.302 INFO:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-05-18T21:34:33.308 INFO:teuthology.orchestra.run.smithi196:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-05-18T21:34:33.325 INFO:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-05-18T21:34:33.333 INFO:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-05-18T21:34:33.340 INFO:teuthology.orchestra.run.smithi080:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-05-18T21:34:33.357 INFO:teuthology.orchestra.run.smithi080:> sudo service rsyslog restart 2020-05-18T21:34:33.400 INFO:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2020-05-18T21:34:33.465 INFO:teuthology.run_tasks:Running task internal.timer... 2020-05-18T21:34:33.485 INFO:teuthology.task.internal:Starting timer... 2020-05-18T21:34:33.485 INFO:teuthology.run_tasks:Running task kernel... 2020-05-18T21:34:33.517 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'mds.a': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mds.b': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}} 2020-05-18T21:34:33.517 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-05-18T21:34:33.519 INFO:teuthology.orchestra.run.smithi196:> uname -r 2020-05-18T21:34:33.526 INFO:teuthology.orchestra.run.smithi196.stdout:5.4.0-29-generic 2020-05-18T21:34:33.526 INFO:teuthology.task.kernel:Running kernel on smithi196: 5.4.0-29-generic 2020-05-18T21:34:33.527 INFO:teuthology.orchestra.run.smithi196:> sudo apt-get clean 2020-05-18T21:34:33.628 INFO:teuthology.orchestra.run.smithi196:> sudo apt-get update 2020-05-18T21:34:34.025 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2020-05-18T21:34:34.025 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2020-05-18T21:34:34.033 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2020-05-18T21:34:34.114 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2020-05-18T21:34:35.953 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2020-05-18T21:34:35.972 INFO:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-05-18T21:34:36.047 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2020-05-18T21:34:36.203 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2020-05-18T21:34:36.205 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2020-05-18T21:34:36.292 INFO:teuthology.orchestra.run.smithi196.stderr:E: Unable to locate package linux-image-current-generic 2020-05-18T21:34:36.294 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-05-18T21:34:36.295 INFO:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-05-18T21:34:36.361 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2020-05-18T21:34:36.495 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2020-05-18T21:34:36.496 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2020-05-18T21:34:36.644 INFO:teuthology.orchestra.run.smithi196.stdout:linux-image-generic is already the newest version (5.4.0.29.34). 2020-05-18T21:34:36.645 INFO:teuthology.orchestra.run.smithi196.stdout:0 upgraded, 0 newly installed, 0 to remove and 50 not upgraded. 2020-05-18T21:34:36.646 INFO:teuthology.orchestra.run.smithi196:> dpkg -s linux-image-generic 2020-05-18T21:34:36.697 INFO:teuthology.orchestra.run.smithi196.stdout:Package: linux-image-generic 2020-05-18T21:34:36.697 INFO:teuthology.orchestra.run.smithi196.stdout:Status: install ok installed 2020-05-18T21:34:36.697 INFO:teuthology.orchestra.run.smithi196.stdout:Priority: optional 2020-05-18T21:34:36.698 INFO:teuthology.orchestra.run.smithi196.stdout:Section: kernel 2020-05-18T21:34:36.698 INFO:teuthology.orchestra.run.smithi196.stdout:Installed-Size: 17 2020-05-18T21:34:36.698 INFO:teuthology.orchestra.run.smithi196.stdout:Maintainer: Ubuntu Kernel Team 2020-05-18T21:34:36.698 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: amd64 2020-05-18T21:34:36.698 INFO:teuthology.orchestra.run.smithi196.stdout:Source: linux-meta 2020-05-18T21:34:36.699 INFO:teuthology.orchestra.run.smithi196.stdout:Version: 5.4.0.29.34 2020-05-18T21:34:36.699 INFO:teuthology.orchestra.run.smithi196.stdout:Provides: virtualbox-guest-modules (= 6.1.6-dfsg-1), wireguard-modules (= 1.0.20200413-1), zfs-modules (= 0.8.3-1ubuntu12) 2020-05-18T21:34:36.699 INFO:teuthology.orchestra.run.smithi196.stdout:Depends: linux-image-5.4.0-29-generic, linux-modules-extra-5.4.0-29-generic, linux-firmware, intel-microcode, amd64-microcode 2020-05-18T21:34:36.699 INFO:teuthology.orchestra.run.smithi196.stdout:Recommends: thermald 2020-05-18T21:34:36.699 INFO:teuthology.orchestra.run.smithi196.stdout:Description: Generic Linux kernel image 2020-05-18T21:34:36.700 INFO:teuthology.orchestra.run.smithi196.stdout: This package will always depend on the latest generic kernel image 2020-05-18T21:34:36.700 INFO:teuthology.orchestra.run.smithi196.stdout: available. 2020-05-18T21:34:36.700 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-05-18T21:34:36.701 INFO:teuthology.orchestra.run.smithi080:> uname -r 2020-05-18T21:34:36.708 INFO:teuthology.orchestra.run.smithi080.stdout:5.4.0-29-generic 2020-05-18T21:34:36.708 INFO:teuthology.task.kernel:Running kernel on smithi080: 5.4.0-29-generic 2020-05-18T21:34:36.709 INFO:teuthology.orchestra.run.smithi080:> sudo apt-get clean 2020-05-18T21:34:36.818 INFO:teuthology.orchestra.run.smithi080:> sudo apt-get update 2020-05-18T21:34:37.107 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2020-05-18T21:34:37.211 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2020-05-18T21:34:37.292 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2020-05-18T21:34:37.373 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2020-05-18T21:34:39.004 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2020-05-18T21:34:39.026 INFO:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-05-18T21:34:39.109 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2020-05-18T21:34:39.259 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2020-05-18T21:34:39.272 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2020-05-18T21:34:39.364 INFO:teuthology.orchestra.run.smithi080.stderr:E: Unable to locate package linux-image-current-generic 2020-05-18T21:34:39.365 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-05-18T21:34:39.366 INFO:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-05-18T21:34:39.432 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2020-05-18T21:34:39.562 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2020-05-18T21:34:39.562 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2020-05-18T21:34:39.711 INFO:teuthology.orchestra.run.smithi080.stdout:linux-image-generic is already the newest version (5.4.0.29.34). 2020-05-18T21:34:39.711 INFO:teuthology.orchestra.run.smithi080.stdout:0 upgraded, 0 newly installed, 0 to remove and 50 not upgraded. 2020-05-18T21:34:39.713 INFO:teuthology.orchestra.run.smithi080:> dpkg -s linux-image-generic 2020-05-18T21:34:39.760 INFO:teuthology.orchestra.run.smithi080.stdout:Package: linux-image-generic 2020-05-18T21:34:39.760 INFO:teuthology.orchestra.run.smithi080.stdout:Status: install ok installed 2020-05-18T21:34:39.760 INFO:teuthology.orchestra.run.smithi080.stdout:Priority: optional 2020-05-18T21:34:39.760 INFO:teuthology.orchestra.run.smithi080.stdout:Section: kernel 2020-05-18T21:34:39.761 INFO:teuthology.orchestra.run.smithi080.stdout:Installed-Size: 17 2020-05-18T21:34:39.761 INFO:teuthology.orchestra.run.smithi080.stdout:Maintainer: Ubuntu Kernel Team 2020-05-18T21:34:39.761 INFO:teuthology.orchestra.run.smithi080.stdout:Architecture: amd64 2020-05-18T21:34:39.761 INFO:teuthology.orchestra.run.smithi080.stdout:Source: linux-meta 2020-05-18T21:34:39.761 INFO:teuthology.orchestra.run.smithi080.stdout:Version: 5.4.0.29.34 2020-05-18T21:34:39.762 INFO:teuthology.orchestra.run.smithi080.stdout:Provides: virtualbox-guest-modules (= 6.1.6-dfsg-1), wireguard-modules (= 1.0.20200413-1), zfs-modules (= 0.8.3-1ubuntu12) 2020-05-18T21:34:39.762 INFO:teuthology.orchestra.run.smithi080.stdout:Depends: linux-image-5.4.0-29-generic, linux-modules-extra-5.4.0-29-generic, linux-firmware, intel-microcode, amd64-microcode 2020-05-18T21:34:39.762 INFO:teuthology.orchestra.run.smithi080.stdout:Recommends: thermald 2020-05-18T21:34:39.762 INFO:teuthology.orchestra.run.smithi080.stdout:Description: Generic Linux kernel image 2020-05-18T21:34:39.762 INFO:teuthology.orchestra.run.smithi080.stdout: This package will always depend on the latest generic kernel image 2020-05-18T21:34:39.763 INFO:teuthology.orchestra.run.smithi080.stdout: available. 2020-05-18T21:34:39.763 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-05-18T21:34:39.763 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-05-18T21:34:39.764 INFO:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-05-18T21:34:39.780 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2020-05-18T21:34:39.875 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2020-05-18T21:34:39.875 INFO:teuthology.orchestra.run.smithi080:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-05-18T21:34:39.891 INFO:teuthology.orchestra.run.smithi080.stdout:ttyS1 2020-05-18T21:34:40.012 INFO:teuthology.run_tasks:Running task pcp... 2020-05-18T21:34:40.075 INFO:teuthology.run_tasks:Running task selinux... 2020-05-18T21:34:40.086 DEBUG:teuthology.task.selinux:Excluding smithi196: OS 'ubuntu' does not support SELinux 2020-05-18T21:34:40.086 DEBUG:teuthology.task.selinux:Excluding smithi080: OS 'ubuntu' does not support SELinux 2020-05-18T21:34:40.087 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-05-18T21:34:40.087 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2020-05-18T21:34:40.087 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-05-18T21:34:40.088 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-05-18T21:34:40.100 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-05-18T21:34:40.101 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-05-18T21:34:40.119 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-05-18T21:34:40.122 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi196.front.sepia.ceph.com,smithi080.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-05-18T21:34:40.510 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-05-18T21:34:41.566 INFO:teuthology.task.ansible.out: [WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-05-18T21:34:44.005 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-05-18T21:34:44.028 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:44 +0000 (0:00:00.077) 0:00:00.077 ************ 2020-05-18T21:34:44.049 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:44 +0000 (0:00:00.020) 0:00:00.097 ************ 2020-05-18T21:34:46.160 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi080.front.sepia.ceph.com] 2020-05-18T21:34:46.228 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] 2020-05-18T21:34:46.265 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:46 +0000 (0:00:02.216) 0:00:02.314 ************ 2020-05-18T21:34:46.336 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:46 +0000 (0:00:00.071) 0:00:02.385 ************ 2020-05-18T21:34:46.716 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-05-18T21:34:46.718 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-05-18T21:34:46.754 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:46 +0000 (0:00:00.417) 0:00:02.803 ************ 2020-05-18T21:34:46.805 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:46 +0000 (0:00:00.050) 0:00:02.854 ************ 2020-05-18T21:34:47.341 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-05-18T21:34:47.343 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} ok: [smithi196.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-05-18T21:34:47.386 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:47 +0000 (0:00:00.580) 0:00:03.434 ************ 2020-05-18T21:34:47.432 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:47 +0000 (0:00:00.045) 0:00:03.480 ************ 2020-05-18T21:34:47.441 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-05-18T21:34:47.443 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:34:47.487 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:47 +0000 (0:00:00.054) 0:00:03.535 ************ 2020-05-18T21:34:47.496 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:34:47.534 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:47 +0000 (0:00:00.048) 0:00:03.583 ************ 2020-05-18T21:34:47.911 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-05-18T21:34:47.913 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-18T21:34:47.954 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:47 +0000 (0:00:00.419) 0:00:04.002 ************ 2020-05-18T21:34:47.963 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-18T21:34:48.032 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:48 +0000 (0:00:00.078) 0:00:04.081 ************ 2020-05-18T21:34:48.916 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-05-18T21:34:48.919 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/cephlab_sudo", "size": 182, "state": "file", "uid": 0} ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/cephlab_sudo", "size": 182, "state": "file", "uid": 0} 2020-05-18T21:34:48.974 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:48 +0000 (0:00:00.942) 0:00:05.023 ************ 2020-05-18T21:34:49.009 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:49 +0000 (0:00:00.035) 0:00:05.058 ************ 2020-05-18T21:34:49.440 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-05-18T21:34:49.442 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-05-18T21:34:49.450 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail 2020-05-18T21:34:49.457 INFO:teuthology.task.ansible.out:.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@ 2020-05-18T21:34:49.459 INFO:teuthology.task.ansible.out:aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-05-18T21:34:49.609 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-05-18T21:34:49.618 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-05-18T21:34:49.781 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:49 +0000 (0:00:00.771) 0:00:05.829 ************ 2020-05-18T21:34:49.814 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:49 +0000 (0:00:00.033) 0:00:05.863 ************ 2020-05-18T21:34:50.796 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi080.front.sepia.ceph.com] 2020-05-18T21:34:50.944 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] 2020-05-18T21:34:50.986 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-05-18T21:34:50.988 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:50 +0000 (0:00:01.172) 0:00:07.036 ************ 2020-05-18T21:34:51.068 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:51 +0000 (0:00:00.081) 0:00:07.117 ************ 2020-05-18T21:34:51.595 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** changed: [smithi080.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.205299", "end": "2020-05-18 21:34:51.562323", "rc": 0, "start": "2020-05-18 21:34:51.357024", "stderr": "", "stderr_lines": [], "stdout": "c86c547126377d9eb959f02d0ec399c86a476654", "stdout_lines": ["c86c547126377d9eb959f02d0ec399c86a476654"]} changed: [smithi196.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.203652", "end": "2020-05-18 21:34:51.556111", "rc": 0, "start": "2020-05-18 21:34:51.352459", "stderr": "", "stderr_lines": [], "stdout": "c86c547126377d9eb959f02d0ec399c86a476654", "stdout_lines": ["c86c547126377d9eb959f02d0ec399c86a476654"]} 2020-05-18T21:34:51.630 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:51 +0000 (0:00:00.562) 0:00:07.679 ************ 2020-05-18T21:34:51.667 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:51 +0000 (0:00:00.036) 0:00:07.716 ************ 2020-05-18T21:34:51.788 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-05-18T21:34:51.790 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.019788", "end": "2020-05-18 21:34:51.374661", "failed_when_result": false, "rc": 0, "start": "2020-05-18 21:34:51.354873", "stderr": "", "stderr_lines": [], "stdout": "6c6e94be874b2e32f2708dec8d44a0e25f99b226", "stdout_lines": ["6c6e94be874b2e32f2708dec8d44a0e25f99b226"]} 2020-05-18T21:34:51.832 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:51 +0000 (0:00:00.165) 0:00:07.881 ************ 2020-05-18T21:34:51.843 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.013466", "end": "2020-05-18 21:34:52.232306", "failed_when_result": false, "rc": 0, "start": "2020-05-18 21:34:52.218840", "stderr": "", "stderr_lines": [], "stdout": "6c6e94be874b2e32f2708dec8d44a0e25f99b226", "stdout_lines": ["6c6e94be874b2e32f2708dec8d44a0e25f99b226"]} 2020-05-18T21:34:51.880 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:51 +0000 (0:00:00.047) 0:00:07.929 ************ 2020-05-18T21:34:51.896 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:34:51.936 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:51 +0000 (0:00:00.056) 0:00:07.985 ************ 2020-05-18T21:34:51.945 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:34:51.983 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:51 +0000 (0:00:00.046) 0:00:08.032 ************ 2020-05-18T21:34:51.991 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-05-18T21:34:51.993 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:34:52.029 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:52 +0000 (0:00:00.045) 0:00:08.077 ************ 2020-05-18T21:34:52.038 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:34:52.076 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:52 +0000 (0:00:00.047) 0:00:08.125 ************ 2020-05-18T21:34:52.084 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-05-18T21:34:52.086 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:34:52.143 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:52 +0000 (0:00:00.066) 0:00:08.192 ************ 2020-05-18T21:34:52.153 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:34:52.200 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:52 +0000 (0:00:00.056) 0:00:08.248 ************ 2020-05-18T21:34:52.208 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-05-18T21:34:52.210 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:34:52.247 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:52 +0000 (0:00:00.047) 0:00:08.296 ************ 2020-05-18T21:34:52.256 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:34:52.299 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:52 +0000 (0:00:00.052) 0:00:08.348 ************ 2020-05-18T21:34:52.309 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-05-18T21:34:52.311 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:34:52.355 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:52 +0000 (0:00:00.056) 0:00:08.404 ************ 2020-05-18T21:34:52.365 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:34:52.419 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:52 +0000 (0:00:00.063) 0:00:08.467 ************ 2020-05-18T21:34:52.501 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-05-18T21:34:52.503 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-18T21:34:52.570 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:52 +0000 (0:00:00.151) 0:00:08.619 ************ 2020-05-18T21:34:52.579 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-18T21:34:52.630 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:34:52 +0000 (0:00:00.059) 0:00:08.679 ************ 2020-05-18T21:34:52.755 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-05-18T21:34:52.757 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2020-05-18T21:34:52.822 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2020-05-18T21:34:52.903 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2020-05-18T21:34:52.970 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2020-05-18T21:34:53.047 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2020-05-18T21:34:53.109 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2020-05-18T21:34:53.183 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2020-05-18T21:34:53.245 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2020-05-18T21:34:53.332 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2020-05-18T21:34:53.390 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2020-05-18T21:34:53.490 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2020-05-18T21:34:53.522 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2020-05-18T21:34:53.631 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2020-05-18T21:34:53.652 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2020-05-18T21:34:53.778 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2020-05-18T21:34:53.791 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2020-05-18T21:34:53.934 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2020-05-18T21:34:53.946 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2020-05-18T21:34:54.079 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2020-05-18T21:34:54.091 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2020-05-18T21:34:54.218 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2020-05-18T21:34:54.247 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2020-05-18T21:34:54.355 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2020-05-18T21:34:54.391 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2020-05-18T21:34:54.503 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2020-05-18T21:34:54.525 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2020-05-18T21:34:54.638 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2020-05-18T21:34:54.677 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2020-05-18T21:34:54.780 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2020-05-18T21:34:54.823 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2020-05-18T21:34:54.928 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2020-05-18T21:34:54.987 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2020-05-18T21:34:55.059 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2020-05-18T21:34:55.142 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2020-05-18T21:34:55.199 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2020-05-18T21:34:55.282 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2020-05-18T21:34:55.346 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2020-05-18T21:34:55.430 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2020-05-18T21:34:55.495 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2020-05-18T21:34:55.570 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2020-05-18T21:34:55.654 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2020-05-18T21:34:55.718 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2020-05-18T21:34:55.815 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1122} 2020-05-18T21:34:55.864 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1122} 2020-05-18T21:34:55.971 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2020-05-18T21:34:56.007 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2020-05-18T21:34:56.112 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2020-05-18T21:34:56.149 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2020-05-18T21:34:56.246 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2020-05-18T21:34:56.292 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2020-05-18T21:34:56.394 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2020-05-18T21:34:56.437 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2020-05-18T21:34:56.525 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2020-05-18T21:34:56.584 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2020-05-18T21:34:56.673 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2020-05-18T21:34:56.728 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2020-05-18T21:34:56.826 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2020-05-18T21:34:56.867 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2020-05-18T21:34:56.953 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2020-05-18T21:34:57.016 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2020-05-18T21:34:57.099 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2020-05-18T21:34:57.162 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2020-05-18T21:34:57.238 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2020-05-18T21:34:57.298 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2020-05-18T21:34:57.370 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2020-05-18T21:34:57.440 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2020-05-18T21:34:57.522 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2020-05-18T21:34:57.600 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2020-05-18T21:34:57.648 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2020-05-18T21:34:57.735 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2020-05-18T21:34:57.787 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2020-05-18T21:34:57.872 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2020-05-18T21:34:57.935 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2020-05-18T21:34:58.030 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2020-05-18T21:34:58.064 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2020-05-18T21:34:58.178 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2020-05-18T21:34:58.200 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-05-18T21:34:58.314 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-05-18T21:34:58.347 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-05-18T21:34:58.467 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-05-18T21:34:58.478 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2020-05-18T21:34:58.605 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2020-05-18T21:34:58.616 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2020-05-18T21:34:58.750 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2020-05-18T21:34:58.769 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2020-05-18T21:34:58.904 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2020-05-18T21:34:58.924 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2020-05-18T21:34:59.042 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2020-05-18T21:34:59.053 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2020-05-18T21:34:59.178 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2020-05-18T21:34:59.190 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2020-05-18T21:34:59.315 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2020-05-18T21:34:59.337 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-05-18T21:34:59.461 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-05-18T21:34:59.472 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-05-18T21:34:59.600 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-05-18T21:34:59.611 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2020-05-18T21:34:59.740 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2020-05-18T21:34:59.751 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2020-05-18T21:34:59.876 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2020-05-18T21:34:59.877 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2020-05-18T21:35:00.027 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2020-05-18T21:35:00.049 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2020-05-18T21:35:00.175 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2020-05-18T21:35:00.197 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2020-05-18T21:35:00.313 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2020-05-18T21:35:00.335 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2020-05-18T21:35:00.467 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2020-05-18T21:35:00.489 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2020-05-18T21:35:00.615 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2020-05-18T21:35:00.804 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2020-05-18T21:35:00.927 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2020-05-18T21:35:00.950 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2020-05-18T21:35:01.086 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2020-05-18T21:35:01.094 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2020-05-18T21:35:01.224 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2020-05-18T21:35:01.235 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2020-05-18T21:35:01.366 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2020-05-18T21:35:01.379 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2020-05-18T21:35:01.539 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2020-05-18T21:35:01.541 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2020-05-18T21:35:01.692 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2020-05-18T21:35:01.693 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2020-05-18T21:35:01.840 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2020-05-18T21:35:01.855 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2020-05-18T21:35:01.857 INFO:teuthology.task.ansible.out: 2020-05-18T21:35:01.979 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-05-18T21:35:02.022 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2020-05-18T21:35:02.024 INFO:teuthology.task.ansible.out: 2020-05-18T21:35:02.108 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2020-05-18T21:35:02.152 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-05-18T21:35:02.256 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2020-05-18T21:35:02.298 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2020-05-18T21:35:02.396 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2020-05-18T21:35:02.443 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2020-05-18T21:35:02.538 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2020-05-18T21:35:02.613 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2020-05-18T21:35:02.698 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2020-05-18T21:35:02.757 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2020-05-18T21:35:02.865 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2020-05-18T21:35:02.920 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2020-05-18T21:35:03.012 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2020-05-18T21:35:03.065 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2020-05-18T21:35:03.147 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2020-05-18T21:35:03.186 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2020-05-18T21:35:03.320 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2020-05-18T21:35:03.353 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2020-05-18T21:35:03.488 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2020-05-18T21:35:03.562 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2020-05-18T21:35:03.626 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2020-05-18T21:35:03.702 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2020-05-18T21:35:03.765 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2020-05-18T21:35:03.855 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2020-05-18T21:35:03.909 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2020-05-18T21:35:04.001 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2020-05-18T21:35:04.054 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2020-05-18T21:35:04.146 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2020-05-18T21:35:04.229 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2020-05-18T21:35:04.284 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2020-05-18T21:35:04.367 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2020-05-18T21:35:04.437 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2020-05-18T21:35:04.515 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2020-05-18T21:35:04.613 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2020-05-18T21:35:04.656 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2020-05-18T21:35:04.760 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2020-05-18T21:35:04.809 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2020-05-18T21:35:04.907 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2020-05-18T21:35:04.949 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2020-05-18T21:35:05.052 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2020-05-18T21:35:05.095 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2020-05-18T21:35:05.195 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2020-05-18T21:35:05.231 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2020-05-18T21:35:05.338 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2020-05-18T21:35:05.371 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2020-05-18T21:35:05.482 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2020-05-18T21:35:05.515 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2020-05-18T21:35:05.627 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2020-05-18T21:35:05.661 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2020-05-18T21:35:05.800 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2020-05-18T21:35:05.935 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2020-05-18T21:35:05.960 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2020-05-18T21:35:05.971 INFO:teuthology.task.ansible.out: 2020-05-18T21:35:06.070 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2020-05-18T21:35:06.130 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2020-05-18T21:35:06.215 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2020-05-18T21:35:06.259 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2020-05-18T21:35:06.393 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2020-05-18T21:35:06.415 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2020-05-18T21:35:06.537 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2020-05-18T21:35:06.559 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2020-05-18T21:35:06.681 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2020-05-18T21:35:06.703 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2020-05-18T21:35:06.832 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2020-05-18T21:35:06.864 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2020-05-18T21:35:06.971 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2020-05-18T21:35:06.993 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2020-05-18T21:35:07.118 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2020-05-18T21:35:07.140 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2020-05-18T21:35:07.254 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2020-05-18T21:35:07.277 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2020-05-18T21:35:07.398 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2020-05-18T21:35:07.434 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2020-05-18T21:35:07.550 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2020-05-18T21:35:07.587 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-05-18T21:35:07.711 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2020-05-18T21:35:07.725 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2020-05-18T21:35:07.853 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-05-18T21:35:07.885 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2020-05-18T21:35:07.994 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2020-05-18T21:35:08.047 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2020-05-18T21:35:08.132 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2020-05-18T21:35:08.196 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2020-05-18T21:35:08.278 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2020-05-18T21:35:08.335 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2020-05-18T21:35:08.420 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2020-05-18T21:35:08.484 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2020-05-18T21:35:08.568 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2020-05-18T21:35:08.641 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2020-05-18T21:35:08.725 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2020-05-18T21:35:08.769 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2020-05-18T21:35:08.859 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2020-05-18T21:35:08.913 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2020-05-18T21:35:09.007 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2020-05-18T21:35:09.050 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2020-05-18T21:35:09.149 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2020-05-18T21:35:09.194 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2020-05-18T21:35:09.293 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2020-05-18T21:35:09.338 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2020-05-18T21:35:09.444 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2020-05-18T21:35:09.583 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2020-05-18T21:35:09.917 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2020-05-18T21:35:10.136 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2020-05-18T21:35:10.411 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2020-05-18T21:35:10.511 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:10 +0000 (0:00:17.881) 0:00:26.560 ************ 2020-05-18T21:35:10.542 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-05-18T21:35:10.582 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:10 +0000 (0:00:00.070) 0:00:26.631 ************ 2020-05-18T21:35:10.610 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-05-18T21:35:10.612 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2020-05-18T21:35:10.680 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-05-18T21:35:10.682 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 2020-05-18T21:35:10.683 INFO:teuthology.task.ansible.out: 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAx 2020-05-18T21:35:10.685 INFO:teuthology.task.ansible.out:j+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8ea 2020-05-18T21:35:10.687 INFO:teuthology.task.ansible.out:c8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f261 2020-05-18T21:35:10.688 INFO:teuthology.task.ansible.out:02d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomai 2020-05-18T21:35:10.690 INFO:teuthology.task.ansible.out:wang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redha 2020-05-18T21:35:10.692 INFO:teuthology.task.ansible.out:t.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMiv 2020-05-18T21:35:10.694 INFO:teuthology.task.ansible.out:DNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad 2020-05-18T21:35:10.694 INFO:teuthology.task.ansible.out:7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "beng 2020-05-18T21:35:10.696 INFO:teuthology.task.ansible.out:land@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name" 2020-05-18T21:35:10.697 INFO:teuthology.task.ansible.out:: "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643 2020-05-18T21:35:10.699 INFO:teuthology.task.ansible.out:eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb 2020-05-18T21:35:10.700 INFO:teuthology.task.ansible.out:801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}]}, "changed": false} 2020-05-18T21:35:10.730 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:10 +0000 (0:00:00.147) 0:00:26.778 ************ 2020-05-18T21:35:10.779 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:10 +0000 (0:00:00.049) 0:00:26.828 ************ 2020-05-18T21:35:10.787 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-05-18T21:35:10.825 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:10 +0000 (0:00:00.045) 0:00:26.874 ************ 2020-05-18T21:35:10.886 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-05-18T21:35:10.890 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 2020-05-18T21:35:10.891 INFO:teuthology.task.ansible.out: 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAx 2020-05-18T21:35:10.893 INFO:teuthology.task.ansible.out:j+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8ea 2020-05-18T21:35:10.895 INFO:teuthology.task.ansible.out:c8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f261 2020-05-18T21:35:10.896 INFO:teuthology.task.ansible.out:02d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomai 2020-05-18T21:35:10.898 INFO:teuthology.task.ansible.out:wang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redha 2020-05-18T21:35:10.900 INFO:teuthology.task.ansible.out:t.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4Uk 2020-05-18T21:35:10.900 INFO:teuthology.task.ansible.out:ibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMiv 2020-05-18T21:35:10.902 INFO:teuthology.task.ansible.out:DNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "beng 2020-05-18T21:35:10.903 INFO:teuthology.task.ansible.out:land@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name" 2020-05-18T21:35:10.905 INFO:teuthology.task.ansible.out:: "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643 2020-05-18T21:35:10.906 INFO:teuthology.task.ansible.out:eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb 2020-05-18T21:35:10.908 INFO:teuthology.task.ansible.out:801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}]}, "changed": false} 2020-05-18T21:35:10.940 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:10 +0000 (0:00:00.115) 0:00:26.989 ************ 2020-05-18T21:35:11.657 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-05-18T21:35:11.659 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com -> localhost] => {"after": "c86c547126377d9eb959f02d0ec399c86a476654", "attempts": 1, "before": "c86c547126377d9eb959f02d0ec399c86a476654", "changed": false, "remote_url_changed": false} 2020-05-18T21:35:11.705 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:11 +0000 (0:00:00.764) 0:00:27.754 ************ 2020-05-18T21:35:11.723 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com -> localhost] => {"after": "c86c547126377d9eb959f02d0ec399c86a476654", "attempts": 1, "before": "c86c547126377d9eb959f02d0ec399c86a476654", "changed": false, "remote_url_changed": false} 2020-05-18T21:35:11.764 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:11 +0000 (0:00:00.059) 0:00:27.813 ************ 2020-05-18T21:35:11.902 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-18T21:35:11.904 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2020-05-18T21:35:11.967 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2020-05-18T21:35:12.125 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2020-05-18T21:35:12.230 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2020-05-18T21:35:12.283 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2020-05-18T21:35:12.435 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2020-05-18T21:35:12.437 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2020-05-18T21:35:12.568 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2020-05-18T21:35:12.593 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2020-05-18T21:35:12.595 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2020-05-18T21:35:12.719 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2020-05-18T21:35:12.721 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2020-05-18T21:35:12.730 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2020-05-18T21:35:12.858 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2020-05-18T21:35:12.869 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2020-05-18T21:35:12.871 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2020-05-18T21:35:12.993 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2020-05-18T21:35:13.005 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2020-05-18T21:35:13.140 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2020-05-18T21:35:13.142 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR 2020-05-18T21:35:13.143 INFO:teuthology.task.ansible.out:1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:13.280 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2020-05-18T21:35:13.281 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2020-05-18T21:35:13.282 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2020-05-18T21:35:13.291 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:13.293 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr 2020-05-18T21:35:13.295 INFO:teuthology.task.ansible.out:8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:13.303 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:13.422 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2020-05-18T21:35:13.424 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2020-05-18T21:35:13.435 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2020-05-18T21:35:13.436 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:13.457 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:13.473 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:13.571 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2020-05-18T21:35:13.572 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2020-05-18T21:35:13.614 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2020-05-18T21:35:13.719 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2020-05-18T21:35:13.759 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2020-05-18T21:35:13.760 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2020-05-18T21:35:13.852 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2020-05-18T21:35:13.895 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2020-05-18T21:35:13.988 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2020-05-18T21:35:14.021 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2020-05-18T21:35:14.124 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2020-05-18T21:35:14.167 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2020-05-18T21:35:14.262 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2020-05-18T21:35:14.325 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2020-05-18T21:35:14.398 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2020-05-18T21:35:14.467 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2020-05-18T21:35:14.539 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2020-05-18T21:35:14.541 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2020-05-18T21:35:14.612 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2020-05-18T21:35:14.676 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2020-05-18T21:35:14.678 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2020-05-18T21:35:14.754 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2020-05-18T21:35:14.756 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2020-05-18T21:35:14.810 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2020-05-18T21:35:14.909 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2020-05-18T21:35:14.911 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2020-05-18T21:35:14.951 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2020-05-18T21:35:15.039 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2020-05-18T21:35:15.101 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2020-05-18T21:35:15.103 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2020-05-18T21:35:15.176 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2020-05-18T21:35:15.249 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2020-05-18T21:35:15.313 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2020-05-18T21:35:15.315 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2020-05-18T21:35:15.422 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2020-05-18T21:35:15.455 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2020-05-18T21:35:15.560 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2020-05-18T21:35:15.602 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2020-05-18T21:35:15.718 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2020-05-18T21:35:15.750 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2020-05-18T21:35:15.867 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2020-05-18T21:35:15.898 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2020-05-18T21:35:16.016 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2020-05-18T21:35:16.018 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2020-05-18T21:35:16.037 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2020-05-18T21:35:16.184 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2020-05-18T21:35:16.186 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2020-05-18T21:35:16.195 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2020-05-18T21:35:16.342 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2020-05-18T21:35:16.363 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2020-05-18T21:35:16.478 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2020-05-18T21:35:16.489 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2020-05-18T21:35:16.614 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2020-05-18T21:35:16.645 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2020-05-18T21:35:16.764 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2020-05-18T21:35:16.786 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2020-05-18T21:35:16.901 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2020-05-18T21:35:16.923 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2020-05-18T21:35:17.038 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2020-05-18T21:35:17.060 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-05-18T21:35:17.174 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-05-18T21:35:17.207 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2020-05-18T21:35:17.336 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2020-05-18T21:35:17.338 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2020-05-18T21:35:17.473 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2020-05-18T21:35:17.475 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2020-05-18T21:35:17.485 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2020-05-18T21:35:17.609 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2020-05-18T21:35:17.611 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2020-05-18T21:35:17.641 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2020-05-18T21:35:17.642 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2020-05-18T21:35:17.745 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2020-05-18T21:35:17.838 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2020-05-18T21:35:17.839 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2020-05-18T21:35:17.904 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2020-05-18T21:35:17.976 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2020-05-18T21:35:18.050 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2020-05-18T21:35:18.051 INFO:teuthology.task.ansible.out: 2020-05-18T21:35:18.123 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2020-05-18T21:35:18.176 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2020-05-18T21:35:18.178 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-05-18T21:35:18.259 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2020-05-18T21:35:18.322 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2020-05-18T21:35:18.324 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2020-05-18T21:35:18.326 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2020-05-18T21:35:18.396 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2020-05-18T21:35:18.398 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-05-18T21:35:18.460 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2020-05-18T21:35:18.545 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2020-05-18T21:35:18.546 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2020-05-18T21:35:18.548 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2020-05-18T21:35:18.589 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2020-05-18T21:35:18.679 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2020-05-18T21:35:18.735 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2020-05-18T21:35:18.818 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2020-05-18T21:35:18.877 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2020-05-18T21:35:18.950 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2020-05-18T21:35:19.013 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2020-05-18T21:35:19.086 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2020-05-18T21:35:19.149 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2020-05-18T21:35:19.151 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2020-05-18T21:35:19.224 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2020-05-18T21:35:19.288 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2020-05-18T21:35:19.362 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2020-05-18T21:35:19.364 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2020-05-18T21:35:19.425 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2020-05-18T21:35:19.488 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2020-05-18T21:35:19.564 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2020-05-18T21:35:19.566 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2020-05-18T21:35:19.628 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2020-05-18T21:35:19.707 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2020-05-18T21:35:19.760 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2020-05-18T21:35:19.761 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2020-05-18T21:35:19.833 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2020-05-18T21:35:19.886 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2020-05-18T21:35:19.990 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2020-05-18T21:35:19.992 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2020-05-18T21:35:20.021 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2020-05-18T21:35:20.123 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2020-05-18T21:35:20.166 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2020-05-18T21:35:20.167 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2020-05-18T21:35:20.261 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2020-05-18T21:35:20.302 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2020-05-18T21:35:20.442 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2020-05-18T21:35:20.464 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2020-05-18T21:35:20.590 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-05-18T21:35:20.601 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2020-05-18T21:35:20.716 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2020-05-18T21:35:20.727 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-05-18T21:35:20.862 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2020-05-18T21:35:20.895 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2020-05-18T21:35:20.989 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2020-05-18T21:35:21.042 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2020-05-18T21:35:21.147 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2020-05-18T21:35:21.213 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2020-05-18T21:35:21.288 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2020-05-18T21:35:21.352 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2020-05-18T21:35:21.429 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2020-05-18T21:35:21.486 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2020-05-18T21:35:21.560 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2020-05-18T21:35:21.629 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2020-05-18T21:35:21.732 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2020-05-18T21:35:21.753 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2020-05-18T21:35:21.868 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2020-05-18T21:35:21.889 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2020-05-18T21:35:22.017 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2020-05-18T21:35:22.042 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2020-05-18T21:35:22.178 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2020-05-18T21:35:22.179 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2020-05-18T21:35:22.317 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2020-05-18T21:35:22.319 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2020-05-18T21:35:22.449 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2020-05-18T21:35:22.471 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2020-05-18T21:35:22.594 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2020-05-18T21:35:22.616 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2020-05-18T21:35:22.735 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2020-05-18T21:35:22.746 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2020-05-18T21:35:22.875 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2020-05-18T21:35:22.922 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2020-05-18T21:35:23.035 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2020-05-18T21:35:23.063 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2020-05-18T21:35:23.168 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2020-05-18T21:35:23.199 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2020-05-18T21:35:23.323 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2020-05-18T21:35:23.339 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2020-05-18T21:35:23.475 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2020-05-18T21:35:23.486 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2020-05-18T21:35:23.608 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2020-05-18T21:35:23.643 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2020-05-18T21:35:23.754 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2020-05-18T21:35:23.804 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2020-05-18T21:35:23.806 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2020-05-18T21:35:23.891 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2020-05-18T21:35:23.894 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2020-05-18T21:35:23.944 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2020-05-18T21:35:23.946 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2020-05-18T21:35:24.040 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2020-05-18T21:35:24.042 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2020-05-18T21:35:24.089 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2020-05-18T21:35:24.203 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2020-05-18T21:35:24.260 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2020-05-18T21:35:24.263 INFO:teuthology.task.ansible.out: 2020-05-18T21:35:24.335 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2020-05-18T21:35:24.429 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2020-05-18T21:35:24.482 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2020-05-18T21:35:24.555 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2020-05-18T21:35:24.618 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2020-05-18T21:35:24.701 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2020-05-18T21:35:24.768 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2020-05-18T21:35:24.852 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2020-05-18T21:35:24.919 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2020-05-18T21:35:25.003 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2020-05-18T21:35:25.060 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2020-05-18T21:35:25.145 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2020-05-18T21:35:25.221 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2020-05-18T21:35:25.285 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2020-05-18T21:35:25.370 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2020-05-18T21:35:25.443 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2020-05-18T21:35:25.549 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2020-05-18T21:35:25.605 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2020-05-18T21:35:25.680 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2020-05-18T21:35:25.743 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2020-05-18T21:35:25.834 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2020-05-18T21:35:25.894 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2020-05-18T21:35:25.988 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2020-05-18T21:35:26.031 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2020-05-18T21:35:26.137 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2020-05-18T21:35:26.170 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2020-05-18T21:35:26.172 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-05-18T21:35:26.282 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2020-05-18T21:35:26.284 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-05-18T21:35:26.304 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2020-05-18T21:35:26.422 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2020-05-18T21:35:26.444 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2020-05-18T21:35:26.565 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2020-05-18T21:35:26.576 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2020-05-18T21:35:26.729 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2020-05-18T21:35:26.742 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2020-05-18T21:35:26.878 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2020-05-18T21:35:26.882 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzVa77kg0qGGE3IpMZgSYJ+UwQiDEOg3Lji470V/sQ/R7g6/v7FZpaYqGn1wG3XeUp2gGACxNKLQfGV9EGU66NvfQYqn9A8dXRpTpZuGWN8EcKK98YrSAq0s0WzeiEzVT86RHzVHIqL6/Zn4B1p9sIeY9Hb+bfKOdtYkRBDoFELtiMRzHPFaiIJFnXl+EhH8wceYqXVNNvCuOUBMnOiDYLDtujvYdPt+ff7rvXuOzbwcAKoINFM1C4FtoUEYriKlsxJ2rr7AblfDm/WIrCZcqRBU4JcudtvZOtHhvN4uL6WgzuzuhIQFyePAeIHFUn6AiwKC/MApq7z1Qj+N3pN/Nz xiubli@fedora1\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkS3qZAsTQMObkUS9QBEpYlL7A3/4ypeRiVQil7UXrZkXVhYd8RAlvSBomOHtslz6OMuvm1GtP53apQQbLb2ziJXlogua/2ju17iyBP8KfyGUzx45S3XF6GaSi3C4gyviQ0hLhgg0HSaO92qw7lbjf731hXYBWXDwkXz/6wbYecykKHTM2xeIhTqCSDIx+fdrHmYW8tXp5ZIGxmCXgLhyNLaMXlNrsGHvLixrnUb3abJebdi+diQxWn4fx2Q+WwoAnY1uwawyeFJzmLEs3qjFWmuYSKcDJ94gHHedXyDhZN1mZOs0a0E319jtXG88/SQrX3+/OypS92Hr0FF6PQwhT root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2020-05-18T21:35:27.036 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2020-05-18T21:35:27.037 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzVa77kg0qGGE3IpMZgSYJ+UwQiDEOg3Lji470V/sQ/R7g6/v7FZpaYqGn1wG3XeUp2gGACxNKLQfGV9EGU66NvfQYqn9A8dXRpTpZuGWN8EcKK98YrSAq0s0WzeiEzVT86RHzVHIqL6/Zn4B1p9sIeY9Hb+bfKOdtYkRBDoFELtiMRzHPFaiIJFnXl+EhH8wceYqXVNNvCuOUBMnOiDYLDtujvYdPt+ff7rvXuOzbwcAKoINFM1C4FtoUEYriKlsxJ2rr7AblfDm/WIrCZcqRBU4JcudtvZOtHhvN4uL6WgzuzuhIQFyePAeIHFUn6AiwKC/MApq7z1Qj+N3pN/Nz xiubli@fedora1\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkS3qZAsTQMObkUS9QBEpYlL7A3/4ypeRiVQil7UXrZkXVhYd8RAlvSBomOHtslz6OMuvm1GtP53apQQbLb2ziJXlogua/2ju17iyBP8KfyGUzx45S3XF6GaSi3C4gyviQ0hLhgg0HSaO92qw7lbjf731hXYBWXDwkXz/6wbYecykKHTM2xeIhTqCSDIx+fdrHmYW8tXp5ZIGxmCXgLhyNLaMXlNrsGHvLixrnUb3abJebdi+diQxWn4fx2Q+WwoAnY1uwawyeFJzmLEs3qjFWmuYSKcDJ94gHHedXyDhZN1mZOs0a0E319jtXG88/SQrX3+/OypS92Hr0FF6PQwhT root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2020-05-18T21:35:27.169 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2020-05-18T21:35:27.171 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2020-05-18T21:35:27.328 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2020-05-18T21:35:27.342 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2020-05-18T21:35:27.465 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2020-05-18T21:35:27.477 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2020-05-18T21:35:27.602 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2020-05-18T21:35:27.634 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpzfLLhjb6WlU3D0D26KAzcSgy9FkG1jnU2X6PqvyGuxyqrfDfLRc6d9A/clStS6GmyKfmevMLFr0Qn17FbPguZJ98Cnx4WwTyRyDSY6M4aoHdhKJgX6zGF7EJxCgb4gJdzlVO8uWD9k1ULV5ShS6z24rkjmxtX8Q/nkDa7w6XThLXQHPXmEJZ7ZsURaMwU+A5K0Mun/RO9W1TRKBzqspvb8D8q/o5R1T4Y6nSgMI8c3fbiuikaMogbPFTzjp154VNBD08kqTpi6JFgWUbhp6Cgpi9NJprsaPLvf1hOBnPvMENy+hkLE7TKnOqFPW6NglbB7igLOlLdqZgNfalX+XSkklZgqrJX/8/eXgGAKE4AvPDerAbBd2X9H2I5P84o5Rs8cDw2vmV5zPPT0SOJP+Ge8eXanekbVxbNI3UrrEEj/o3vQz2/ng1EpknlySpy8FIEDKF1qZ/M9Gku0RcquEZTYsUlGkxl1RYh6VhSp2W6QSeAl4B8CUedv4zQ0Cqr18= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2020-05-18T21:35:27.735 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpzfLLhjb6WlU3D0D26KAzcSgy9FkG1jnU2X6PqvyGuxyqrfDfLRc6d9A/clStS6GmyKfmevMLFr0Qn17FbPguZJ98Cnx4WwTyRyDSY6M4aoHdhKJgX6zGF7EJxCgb4gJdzlVO8uWD9k1ULV5ShS6z24rkjmxtX8Q/nkDa7w6XThLXQHPXmEJZ7ZsURaMwU+A5K0Mun/RO9W1TRKBzqspvb8D8q/o5R1T4Y6nSgMI8c3fbiuikaMogbPFTzjp154VNBD08kqTpi6JFgWUbhp6Cgpi9NJprsaPLvf1hOBnPvMENy+hkLE7TKnOqFPW6NglbB7igLOlLdqZgNfalX+XSkklZgqrJX/8/eXgGAKE4AvPDerAbBd2X9H2I5P84o5Rs8cDw2vmV5zPPT0SOJP+Ge8eXanekbVxbNI3UrrEEj/o3vQz2/ng1EpknlySpy8FIEDKF1qZ/M9Gku0RcquEZTYsUlGkxl1RYh6VhSp2W6QSeAl4B8CUedv4zQ0Cqr18= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2020-05-18T21:35:27.779 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2020-05-18T21:35:27.894 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2020-05-18T21:35:27.925 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2020-05-18T21:35:28.040 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2020-05-18T21:35:28.082 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2020-05-18T21:35:28.187 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2020-05-18T21:35:28.229 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2020-05-18T21:35:28.331 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:28 +0000 (0:00:16.565) 0:00:44.378 ************ 2020-05-18T21:35:28.349 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2020-05-18T21:35:28.436 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:28 +0000 (0:00:00.106) 0:00:44.485 ************ 2020-05-18T21:35:28.445 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-18T21:35:28.447 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.448 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.449 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr 2020-05-18T21:35:28.451 INFO:teuthology.task.ansible.out:+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_ 2020-05-18T21:35:28.452 INFO:teuthology.task.ansible.out:loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.463 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.485 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.496 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.507 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.509 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.518 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.529 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.533 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.540 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.551 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.619 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2020-05-18T21:35:28.630 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.694 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2020-05-18T21:35:28.705 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:28.748 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2020-05-18T21:35:28.843 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2020-05-18T21:35:28.890 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2020-05-18T21:35:28.980 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2020-05-18T21:35:29.048 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2020-05-18T21:35:29.069 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.080 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.082 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.096 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.107 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.109 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.118 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.120 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.130 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2020-05-18T21:35:29.132 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.142 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.143 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.154 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.155 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.155 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.180 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.182 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich 2020-05-18T21:35:29.184 INFO:teuthology.task.ansible.out:", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.193 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.195 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.205 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.207 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.226 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.228 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.238 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.239 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.241 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.243 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.246 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.263 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@ 2020-05-18T21:35:29.268 INFO:teuthology.task.ansible.out:desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.271 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.274 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.277 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.288 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.299 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.302 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.302 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn 2020-05-18T21:35:29.304 INFO:teuthology.task.ansible.out:': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.313 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.315 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.325 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.327 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.338 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.349 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.351 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.352 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.353 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.354 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.369 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.384 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.386 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.396 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.398 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.398 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.399 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", 2020-05-18T21:35:29.401 INFO:teuthology.task.ansible.out: "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.403 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.414 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.416 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.426 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.428 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.439 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.440 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.458 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.460 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.464 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.465 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.477 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.478 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.489 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.491 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.501 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.504 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.512 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.515 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.524 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.526 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.527 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.538 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.549 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.551 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.564 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.599 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.601 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.601 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.602 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjk 2020-05-18T21:35:29.603 INFO:teuthology.task.ansible.out:Asj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46 2020-05-18T21:35:29.605 INFO:teuthology.task.ansible.out:e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.606 INFO:teuthology.task.ansible.out: 2020-05-18T21:35:29.619 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.621 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.622 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.633 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.634 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.645 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.646 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.658 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.659 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.669 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.671 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.682 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.684 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.684 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.696 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.698 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.708 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.709 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.721 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.734 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.736 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.746 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.748 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.758 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.760 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.770 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.771 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.771 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.782 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.784 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.794 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.796 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.810 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.817 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.819 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.821 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.829 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.831 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.840 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.842 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.853 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.856 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.857 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.868 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.869 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.869 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.880 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.891 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.892 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.894 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.904 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.915 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.917 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.933 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.934 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.935 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.975 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:29 +0000 (0:00:01.538) 0:00:46.023 ************ 2020-05-18T21:35:29.989 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:29.990 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:30.002 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:30.003 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:30.013 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-05-18T21:35:30.014 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:30.054 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:30 +0000 (0:00:00.079) 0:00:46.102 ************ 2020-05-18T21:35:30.065 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:30.069 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:30.078 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:30.131 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:30 +0000 (0:00:00.077) 0:00:46.180 ************ 2020-05-18T21:35:30.163 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-05-18T21:35:30.164 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:30.202 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:30 +0000 (0:00:00.070) 0:00:46.250 ************ 2020-05-18T21:35:30.257 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-18T21:35:30.259 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2020-05-18T21:35:30.379 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2020-05-18T21:35:30.381 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2020-05-18T21:35:30.502 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2020-05-18T21:35:30.504 INFO:teuthology.task.ansible.out: 2020-05-18T21:35:30.536 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2020-05-18T21:35:30.538 INFO:teuthology.task.ansible.out: 2020-05-18T21:35:30.648 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2020-05-18T21:35:30.669 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2020-05-18T21:35:30.774 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2020-05-18T21:35:30.811 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2020-05-18T21:35:30.909 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2020-05-18T21:35:30.931 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2020-05-18T21:35:31.045 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2020-05-18T21:35:31.099 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2020-05-18T21:35:31.181 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2020-05-18T21:35:31.237 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2020-05-18T21:35:31.311 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2020-05-18T21:35:31.394 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2020-05-18T21:35:31.439 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2020-05-18T21:35:31.554 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2020-05-18T21:35:31.575 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2020-05-18T21:35:31.691 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2020-05-18T21:35:31.702 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2020-05-18T21:35:31.835 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2020-05-18T21:35:31.846 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2020-05-18T21:35:31.986 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2020-05-18T21:35:31.987 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2020-05-18T21:35:32.105 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2020-05-18T21:35:32.238 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} ok: [smithi080.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2020-05-18T21:35:32.323 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2020-05-18T21:35:32.438 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2020-05-18T21:35:32.440 INFO:teuthology.task.ansible.out: 2020-05-18T21:35:32.492 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2020-05-18T21:35:32.590 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2020-05-18T21:35:32.656 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2020-05-18T21:35:32.723 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2020-05-18T21:35:32.800 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2020-05-18T21:35:32.864 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2020-05-18T21:35:32.866 INFO:teuthology.task.ansible.out: 2020-05-18T21:35:32.949 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2020-05-18T21:35:32.992 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2020-05-18T21:35:32.993 INFO:teuthology.task.ansible.out: 2020-05-18T21:35:33.079 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2020-05-18T21:35:33.134 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2020-05-18T21:35:33.210 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2020-05-18T21:35:33.284 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2020-05-18T21:35:33.350 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2020-05-18T21:35:33.463 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2020-05-18T21:35:33.488 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2020-05-18T21:35:33.584 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2020-05-18T21:35:33.623 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2020-05-18T21:35:33.727 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2020-05-18T21:35:33.759 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2020-05-18T21:35:33.867 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2020-05-18T21:35:33.899 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2020-05-18T21:35:33.997 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2020-05-18T21:35:34.032 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2020-05-18T21:35:34.117 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2020-05-18T21:35:34.191 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2020-05-18T21:35:34.275 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2020-05-18T21:35:34.318 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2020-05-18T21:35:34.412 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2020-05-18T21:35:34.454 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2020-05-18T21:35:34.544 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2020-05-18T21:35:34.581 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2020-05-18T21:35:34.687 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2020-05-18T21:35:34.719 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2020-05-18T21:35:34.846 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2020-05-18T21:35:34.867 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2020-05-18T21:35:35.010 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2020-05-18T21:35:35.021 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2020-05-18T21:35:35.148 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2020-05-18T21:35:35.159 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2020-05-18T21:35:35.284 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2020-05-18T21:35:35.295 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2020-05-18T21:35:35.410 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2020-05-18T21:35:35.447 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2020-05-18T21:35:35.540 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2020-05-18T21:35:35.572 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2020-05-18T21:35:35.664 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2020-05-18T21:35:35.723 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2020-05-18T21:35:35.807 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2020-05-18T21:35:35.848 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2020-05-18T21:35:35.945 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2020-05-18T21:35:35.977 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2020-05-18T21:35:36.081 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2020-05-18T21:35:36.132 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2020-05-18T21:35:36.215 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2020-05-18T21:35:36.270 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:36 +0000 (0:00:06.068) 0:00:52.319 ************ 2020-05-18T21:35:36.290 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2020-05-18T21:35:36.346 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:36 +0000 (0:00:00.075) 0:00:52.395 ************ 2020-05-18T21:35:36.909 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-05-18T21:35:36.911 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"changed": true, "checksum": "be49dc3debef3900f7f5b6bdc921fc8b79dea0d7", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "def3daffe56c9db157a75c68cb6f3184", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1589837736.3761075-268189708891589/source", "state": "file", "uid": 0} 2020-05-18T21:35:36.953 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:36 +0000 (0:00:00.606) 0:00:53.002 ************ 2020-05-18T21:35:36.963 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"changed": true, "checksum": "be49dc3debef3900f7f5b6bdc921fc8b79dea0d7", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "def3daffe56c9db157a75c68cb6f3184", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1589837736.3024752-129124824737733/source", "state": "file", "uid": 0} 2020-05-18T21:35:37.011 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.058) 0:00:53.060 ************ 2020-05-18T21:35:37.020 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-05-18T21:35:37.022 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => { "msg": "Host smithi196.front.sepia.ceph.com is running Ubuntu 20.04 (focal)" } 2020-05-18T21:35:37.065 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.053) 0:00:53.114 ************ 2020-05-18T21:35:37.074 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => { "msg": "Host smithi080.front.sepia.ceph.com is running Ubuntu 20.04 (focal)" } 2020-05-18T21:35:37.117 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.051) 0:00:53.165 ************ 2020-05-18T21:35:37.126 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-05-18T21:35:37.128 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-05-18T21:35:37.170 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.048) 0:00:53.214 ************ 2020-05-18T21:35:37.173 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-05-18T21:35:37.211 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.045) 0:00:53.260 ************ 2020-05-18T21:35:37.220 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-05-18T21:35:37.222 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.264 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.052) 0:00:53.313 ************ 2020-05-18T21:35:37.273 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.310 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.046) 0:00:53.359 ************ 2020-05-18T21:35:37.319 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-05-18T21:35:37.321 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.365 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.053) 0:00:53.413 ************ 2020-05-18T21:35:37.374 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.411 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.047) 0:00:53.460 ************ 2020-05-18T21:35:37.421 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-05-18T21:35:37.423 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.462 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.050) 0:00:53.511 ************ 2020-05-18T21:35:37.471 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.510 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.048) 0:00:53.559 ************ 2020-05-18T21:35:37.518 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-05-18T21:35:37.520 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.558 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.048) 0:00:53.607 ************ 2020-05-18T21:35:37.575 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.616 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.058) 0:00:53.665 ************ 2020-05-18T21:35:37.627 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-05-18T21:35:37.628 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.668 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.051) 0:00:53.717 ************ 2020-05-18T21:35:37.677 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.718 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.049) 0:00:53.767 ************ 2020-05-18T21:35:37.727 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-05-18T21:35:37.729 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.768 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.050) 0:00:53.817 ************ 2020-05-18T21:35:37.786 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.822 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.053) 0:00:53.871 ************ 2020-05-18T21:35:37.831 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-05-18T21:35:37.832 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.868 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.046) 0:00:53.917 ************ 2020-05-18T21:35:37.877 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.915 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.046) 0:00:53.964 ************ 2020-05-18T21:35:37.923 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-05-18T21:35:37.925 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:37.961 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:37 +0000 (0:00:00.045) 0:00:54.009 ************ 2020-05-18T21:35:37.974 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.009 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.048) 0:00:54.058 ************ 2020-05-18T21:35:38.017 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-05-18T21:35:38.018 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi196.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:38.055 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.046) 0:00:54.104 ************ 2020-05-18T21:35:38.063 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:38.064 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:38.101 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.045) 0:00:54.150 ************ 2020-05-18T21:35:38.110 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.148 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.046) 0:00:54.196 ************ 2020-05-18T21:35:38.156 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.198 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.050) 0:00:54.247 ************ 2020-05-18T21:35:38.206 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-05-18T21:35:38.208 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.243 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.045) 0:00:54.292 ************ 2020-05-18T21:35:38.252 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.294 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.050) 0:00:54.343 ************ 2020-05-18T21:35:38.306 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi196.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:38.348 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.053) 0:00:54.397 ************ 2020-05-18T21:35:38.357 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:38.398 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.050) 0:00:54.447 ************ 2020-05-18T21:35:38.408 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-05-18T21:35:38.410 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.453 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.054) 0:00:54.502 ************ 2020-05-18T21:35:38.462 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.510 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.056) 0:00:54.558 ************ 2020-05-18T21:35:38.519 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi196.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:38.556 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.046) 0:00:54.605 ************ 2020-05-18T21:35:38.566 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:38.608 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.051) 0:00:54.657 ************ 2020-05-18T21:35:38.618 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-05-18T21:35:38.620 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.668 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.050) 0:00:54.708 ************ 2020-05-18T21:35:38.669 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.708 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.049) 0:00:54.757 ************ 2020-05-18T21:35:38.722 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.757 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.049) 0:00:54.806 ************ 2020-05-18T21:35:38.767 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.809 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.052) 0:00:54.858 ************ 2020-05-18T21:35:38.819 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-05-18T21:35:38.821 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.858 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.048) 0:00:54.907 ************ 2020-05-18T21:35:38.870 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.906 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.047) 0:00:54.955 ************ 2020-05-18T21:35:38.916 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-05-18T21:35:38.918 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:38.955 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:38 +0000 (0:00:00.048) 0:00:55.004 ************ 2020-05-18T21:35:38.964 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.010 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.053) 0:00:55.057 ************ 2020-05-18T21:35:39.018 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-05-18T21:35:39.020 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:39.058 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.049) 0:00:55.106 ************ 2020-05-18T21:35:39.068 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:39.107 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.049) 0:00:55.156 ************ 2020-05-18T21:35:39.124 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-05-18T21:35:39.125 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.166 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.058) 0:00:55.214 ************ 2020-05-18T21:35:39.175 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.218 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.051) 0:00:55.266 ************ 2020-05-18T21:35:39.227 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-05-18T21:35:39.229 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.276 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.059) 0:00:55.325 ************ 2020-05-18T21:35:39.285 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.330 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.053) 0:00:55.379 ************ 2020-05-18T21:35:39.338 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-05-18T21:35:39.340 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.378 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.047) 0:00:55.426 ************ 2020-05-18T21:35:39.386 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.429 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.051) 0:00:55.478 ************ 2020-05-18T21:35:39.438 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-05-18T21:35:39.440 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.476 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.047) 0:00:55.525 ************ 2020-05-18T21:35:39.485 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.527 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.049) 0:00:55.575 ************ 2020-05-18T21:35:39.535 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-05-18T21:35:39.537 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.573 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.046) 0:00:55.622 ************ 2020-05-18T21:35:39.582 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.619 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.045) 0:00:55.668 ************ 2020-05-18T21:35:39.628 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-05-18T21:35:39.630 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.669 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.049) 0:00:55.718 ************ 2020-05-18T21:35:39.678 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.717 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.047) 0:00:55.765 ************ 2020-05-18T21:35:39.727 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-05-18T21:35:39.728 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:39.768 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.051) 0:00:55.817 ************ 2020-05-18T21:35:39.778 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:39.818 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.050) 0:00:55.867 ************ 2020-05-18T21:35:39.827 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-05-18T21:35:39.829 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.867 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.048) 0:00:55.916 ************ 2020-05-18T21:35:39.877 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.914 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.046) 0:00:55.963 ************ 2020-05-18T21:35:39.924 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-05-18T21:35:39.925 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:39.965 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:39 +0000 (0:00:00.049) 0:00:56.012 ************ 2020-05-18T21:35:39.973 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.013 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.049) 0:00:56.062 ************ 2020-05-18T21:35:40.027 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-05-18T21:35:40.029 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.065 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.052) 0:00:56.114 ************ 2020-05-18T21:35:40.074 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.113 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.047) 0:00:56.162 ************ 2020-05-18T21:35:40.121 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-05-18T21:35:40.123 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.123 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.163 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.049) 0:00:56.211 ************ 2020-05-18T21:35:40.182 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.183 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.227 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.064) 0:00:56.276 ************ 2020-05-18T21:35:40.234 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-05-18T21:35:40.238 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.274 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.046) 0:00:56.323 ************ 2020-05-18T21:35:40.283 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.285 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.321 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.047) 0:00:56.370 ************ 2020-05-18T21:35:40.330 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-05-18T21:35:40.332 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.369 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.046) 0:00:56.417 ************ 2020-05-18T21:35:40.376 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.414 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.046) 0:00:56.463 ************ 2020-05-18T21:35:40.541 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-05-18T21:35:40.542 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.017844", "end": "2020-05-18 21:35:40.948071", "rc": 0, "start": "2020-05-18 21:35:40.930227", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-05-18T21:35:40.597 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.178) 0:00:56.642 ************ 2020-05-18T21:35:40.602 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.018708", "end": "2020-05-18 21:35:40.168182", "rc": 0, "start": "2020-05-18 21:35:40.149474", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-05-18T21:35:40.650 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.049) 0:00:56.692 ************ 2020-05-18T21:35:40.651 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-05-18T21:35:40.652 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.689 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.046) 0:00:56.738 ************ 2020-05-18T21:35:40.696 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.733 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.043) 0:00:56.782 ************ 2020-05-18T21:35:40.741 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-05-18T21:35:40.743 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.778 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.045) 0:00:56.827 ************ 2020-05-18T21:35:40.785 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.826 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.048) 0:00:56.875 ************ 2020-05-18T21:35:40.835 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-05-18T21:35:40.837 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.873 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.046) 0:00:56.922 ************ 2020-05-18T21:35:40.880 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.918 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.044) 0:00:56.966 ************ 2020-05-18T21:35:40.925 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-05-18T21:35:40.927 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:40.963 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:40 +0000 (0:00:00.046) 0:00:57.012 ************ 2020-05-18T21:35:40.971 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:41.010 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:41 +0000 (0:00:00.046) 0:00:57.058 ************ 2020-05-18T21:35:41.018 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-05-18T21:35:41.020 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:41.059 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:41 +0000 (0:00:00.049) 0:00:57.108 ************ 2020-05-18T21:35:41.066 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:41.103 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:41 +0000 (0:00:00.044) 0:00:57.152 ************ 2020-05-18T21:35:41.111 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-05-18T21:35:41.114 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:41.149 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:41 +0000 (0:00:00.045) 0:00:57.197 ************ 2020-05-18T21:35:41.157 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:41.194 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:41 +0000 (0:00:00.045) 0:00:57.243 ************ 2020-05-18T21:35:41.203 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-05-18T21:35:41.205 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:41.241 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:41 +0000 (0:00:00.046) 0:00:57.290 ************ 2020-05-18T21:35:41.257 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:41.301 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:41 +0000 (0:00:00.059) 0:00:57.350 ************ 2020-05-18T21:35:41.915 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-05-18T21:35:41.917 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.473006", "end": "2020-05-18 21:35:41.500427", "rc": 0, "start": "2020-05-18 21:35:41.027421", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:35:41.951 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:41 +0000 (0:00:00.650) 0:00:58.000 ************ 2020-05-18T21:35:41.980 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-05-18T21:35:41.982 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:42.016 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:42 +0000 (0:00:00.064) 0:00:58.064 ************ 2020-05-18T21:35:42.089 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-05-18T21:35:42.091 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.702247", "end": "2020-05-18 21:35:42.500406", "rc": 0, "start": "2020-05-18 21:35:41.798159", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:35:42.126 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:42 +0000 (0:00:00.110) 0:00:58.175 ************ 2020-05-18T21:35:42.154 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-05-18T21:35:42.156 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:42.190 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:42 +0000 (0:00:00.063) 0:00:58.239 ************ 2020-05-18T21:35:42.198 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-05-18T21:35:42.199 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. 2020-05-18T21:35:42.200 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.017813", "end": "2020-05-18 21:35:41.761787", "rc": 0, "start": "2020-05-18 21:35:41.743974", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:35:42.237 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:42 +0000 (0:00:00.046) 0:00:58.285 ************ 2020-05-18T21:35:42.361 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.020974", "end": "2020-05-18 21:35:42.758799", "rc": 0, "start": "2020-05-18 21:35:42.737825", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:35:42.398 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:42 +0000 (0:00:00.161) 0:00:58.447 ************ 2020-05-18T21:35:44.679 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-05-18T21:35:44.681 INFO:teuthology.task.ansible.out: [WARNING]: Could not find aptitude. Using apt-get instead ok: [smithi196.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1589837434, "cache_updated": false, "changed": false} 2020-05-18T21:35:44.720 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:44 +0000 (0:00:02.322) 0:01:00.769 ************ 2020-05-18T21:35:44.771 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1589837492, "cache_updated": false, "changed": false} 2020-05-18T21:35:44.810 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:44 +0000 (0:00:00.089) 0:01:00.859 ************ 2020-05-18T21:35:45.364 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-05-18T21:35:45.366 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"cache_update_time": 1589837434, "cache_updated": false, "changed": false} 2020-05-18T21:35:45.402 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:45 +0000 (0:00:00.591) 0:01:01.450 ************ 2020-05-18T21:35:45.431 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:45.474 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:45 +0000 (0:00:00.072) 0:01:01.523 ************ 2020-05-18T21:35:45.483 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-05-18T21:35:45.483 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"cache_update_time": 1589837492, "cache_updated": false, "changed": false} 2020-05-18T21:35:45.522 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:45 +0000 (0:00:00.047) 0:01:01.571 ************ 2020-05-18T21:35:45.558 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:45.600 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:45 +0000 (0:00:00.077) 0:01:01.649 ************ 2020-05-18T21:35:45.836 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-05-18T21:35:45.838 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-05-18T21:35:45.891 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:45 +0000 (0:00:00.291) 0:01:01.940 ************ 2020-05-18T21:35:45.964 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-05-18T21:35:46.036 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:46 +0000 (0:00:00.144) 0:01:02.085 ************ 2020-05-18T21:35:46.530 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-05-18T21:35:46.532 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"cache_update_time": 1589837434, "cache_updated": false, "changed": false} 2020-05-18T21:35:46.578 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:46 +0000 (0:00:00.541) 0:01:02.627 ************ 2020-05-18T21:35:46.650 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"cache_update_time": 1589837492, "cache_updated": false, "changed": false} 2020-05-18T21:35:46.690 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:46 +0000 (0:00:00.112) 0:01:02.739 ************ 2020-05-18T21:35:46.987 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-05-18T21:35:46.988 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-05-18T21:35:47.085 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-05-18T21:35:47.346 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-05-18T21:35:47.420 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-05-18T21:35:47.664 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-05-18T21:35:47.710 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:47 +0000 (0:00:01.019) 0:01:03.758 ************ 2020-05-18T21:35:47.749 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-05-18T21:35:47.799 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:47 +0000 (0:00:00.088) 0:01:03.847 ************ 2020-05-18T21:35:47.845 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-05-18T21:35:47.847 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-05-18T21:35:47.897 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:47 +0000 (0:00:00.098) 0:01:03.946 ************ 2020-05-18T21:35:47.941 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-05-18T21:35:47.979 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:47 +0000 (0:00:00.081) 0:01:04.028 ************ 2020-05-18T21:35:48.262 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-05-18T21:35:48.266 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-05-18T21:35:48.347 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-05-18T21:35:48.563 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-05-18T21:35:48.683 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-05-18T21:35:48.888 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-05-18T21:35:48.927 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:48 +0000 (0:00:00.947) 0:01:04.975 ************ 2020-05-18T21:35:48.971 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-05-18T21:35:48.973 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-05-18T21:35:49.009 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:49 +0000 (0:00:00.081) 0:01:05.057 ************ 2020-05-18T21:35:49.017 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-05-18T21:35:49.018 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-05-18T21:35:49.056 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:49 +0000 (0:00:00.047) 0:01:05.105 ************ 2020-05-18T21:35:49.066 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-05-18T21:35:49.067 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:49.115 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:49 +0000 (0:00:00.058) 0:01:05.163 ************ 2020-05-18T21:35:49.125 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-05-18T21:35:49.127 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-05-18T21:35:49.175 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:49 +0000 (0:00:00.060) 0:01:05.223 ************ 2020-05-18T21:35:49.184 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-05-18T21:35:49.185 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:49.227 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:49 +0000 (0:00:00.052) 0:01:05.276 ************ 2020-05-18T21:35:49.237 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-05-18T21:35:49.239 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:49.275 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:49 +0000 (0:00:00.048) 0:01:05.324 ************ 2020-05-18T21:35:49.285 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-05-18T21:35:49.287 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-18T21:35:49.287 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-18T21:35:49.288 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-18T21:35:49.327 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:49 +0000 (0:00:00.051) 0:01:05.376 ************ 2020-05-18T21:35:49.336 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-05-18T21:35:49.338 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:49.377 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:49 +0000 (0:00:00.049) 0:01:05.425 ************ 2020-05-18T21:35:49.386 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-05-18T21:35:49.388 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:49.427 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:49 +0000 (0:00:00.050) 0:01:05.476 ************ 2020-05-18T21:35:49.437 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-05-18T21:35:49.439 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-18T21:35:49.439 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-18T21:35:49.440 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-18T21:35:49.440 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-18T21:35:49.477 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:49 +0000 (0:00:00.050) 0:01:05.526 ************ 2020-05-18T21:35:49.518 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-05-18T21:35:49.520 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:49.554 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:49 +0000 (0:00:00.076) 0:01:05.603 ************ 2020-05-18T21:35:50.073 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-05-18T21:35:50.075 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"cache_update_time": 1589837434, "cache_updated": false, "changed": false} 2020-05-18T21:35:50.112 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:50 +0000 (0:00:00.557) 0:01:06.161 ************ 2020-05-18T21:35:50.184 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"cache_update_time": 1589837492, "cache_updated": false, "changed": false} 2020-05-18T21:35:50.226 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:50 +0000 (0:00:00.114) 0:01:06.275 ************ 2020-05-18T21:35:50.490 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-05-18T21:35:50.492 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-05-18T21:35:50.531 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:50 +0000 (0:00:00.304) 0:01:06.579 ************ 2020-05-18T21:35:50.592 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-05-18T21:35:50.632 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:50 +0000 (0:00:00.100) 0:01:06.680 ************ 2020-05-18T21:35:50.911 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-05-18T21:35:50.913 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-05-18T21:35:50.953 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:50 +0000 (0:00:00.320) 0:01:07.001 ************ 2020-05-18T21:35:51.015 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-05-18T21:35:51.063 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:51 +0000 (0:00:00.101) 0:01:07.112 ************ 2020-05-18T21:35:51.115 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-05-18T21:35:51.117 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-05-18T21:35:51.222 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-05-18T21:35:51.233 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-05-18T21:35:51.291 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:51 +0000 (0:00:00.228) 0:01:07.340 ************ 2020-05-18T21:35:51.321 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-05-18T21:35:51.323 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:51.360 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:51 +0000 (0:00:00.068) 0:01:07.409 ************ 2020-05-18T21:35:51.369 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-05-18T21:35:51.371 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-05-18T21:35:51.412 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:51 +0000 (0:00:00.052) 0:01:07.461 ************ 2020-05-18T21:35:51.422 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-05-18T21:35:51.424 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:51.461 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:51 +0000 (0:00:00.048) 0:01:07.510 ************ 2020-05-18T21:35:51.470 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-05-18T21:35:51.472 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:51.508 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:51 +0000 (0:00:00.047) 0:01:07.557 ************ 2020-05-18T21:35:51.516 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-05-18T21:35:51.518 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:51.556 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:51 +0000 (0:00:00.047) 0:01:07.605 ************ 2020-05-18T21:35:51.564 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-05-18T21:35:51.565 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-18T21:35:51.601 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:51 +0000 (0:00:00.045) 0:01:07.650 ************ 2020-05-18T21:35:51.641 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-05-18T21:35:51.642 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:51.685 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:51 +0000 (0:00:00.083) 0:01:07.734 ************ 2020-05-18T21:35:51.918 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "9c3d1d79583cf76a6d81bb2ec150bc4af5ca3c5c", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1166, "state": "file", "uid": 0} 2020-05-18T21:35:51.957 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:51 +0000 (0:00:00.271) 0:01:08.006 ************ 2020-05-18T21:35:52.069 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "9c3d1d79583cf76a6d81bb2ec150bc4af5ca3c5c", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1166, "state": "file", "uid": 0} 2020-05-18T21:35:52.109 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:52 +0000 (0:00:00.152) 0:01:08.158 ************ 2020-05-18T21:35:52.721 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-05-18T21:35:52.722 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-05-18 21:34:07 UTC", "ActiveEnterTimestampMonotonic": "16165819", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-journald.socket nss-lookup.target time-sync.target network.target remote-fs.target sysinit.target basic.target -.mount systemd-tmpfiles-setup.service local-fs.target system.slice var-run.mount", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-05-18 21:34:07 UTC", "AssertTimestampMonotonic": "16138107", "Before": "xdm.service shutdown.target multi-user.target plymouth-quit.service getty@tty1.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-05-18 21:34:07 UTC", "Co 2020-05-18T21:35:52.724 INFO:teuthology.task.ansible.out:nditionTimestampMonotonic": "16138106", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1093", "ExecMainStartTimestamp": "Mon 2020-05-18 21:34:07 UTC", "ExecMainStartTimestampMonotonic": "16165637", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Mon 2020-05-18 21:34:07 UTC] ; stop_time=[n/a] ; pid=1093 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; flags= ; start_time=[Mon 2020-05-18 21:34:07 UTC] ; stop_time=[n/a] ; pid=1093 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPostEx": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", 2020-05-18T21:35:52.726 INFO:teuthology.task.ansible.out:"GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-05-18 21:34:07 UTC", "InactiveExitTimestampMonotonic": "16165819", "InvocationID": "f5622b09ef594f5d9f6cc44aeb479a59", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127783", "LimitNPROCSoft": "127783", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127783", "LimitSIGPENDINGSoft": "127783", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "1093", "MemoryAccounting": "yes", "MemoryCurrent": "925696", "Memo 2020-05-18T21:35:52.727 INFO:teuthology.task.ansible.out:ryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/run/nagios/nrpe.pid", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-05-18 21:34:07 UTC", "StateChangeTimestampMonotonic": "16165819", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": " 2020-05-18T21:35:52.729 INFO:teuthology.task.ansible.out:no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "38334", "TimeoutAbortUSec": "1min", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "110", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-18T21:35:52.759 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:52 +0000 (0:00:00.649) 0:01:08.808 ************ 2020-05-18T21:35:52.770 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-05-18 21:34:14 UTC", "ActiveEnterTimestampMonotonic": "16646644", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service -.mount nss-lookup.target local-fs.target time-sync.target remote-fs.target system.slice basic.target var-run.mount systemd-journald.socket sysinit.target network.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-05-18 21:34:14 UTC", "AssertTimestampMonotonic": "16640287", "Before": "plymouth-quit.service multi-user.target getty@tty1.service xdm.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-05-18 21:34:14 UTC", "Co 2020-05-18T21:35:52.772 INFO:teuthology.task.ansible.out:nditionTimestampMonotonic": "16640286", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1105", "ExecMainStartTimestamp": "Mon 2020-05-18 21:34:14 UTC", "ExecMainStartTimestampMonotonic": "16646414", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Mon 2020-05-18 21:34:14 UTC] ; stop_time=[n/a] ; pid=1105 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; flags= ; start_time=[Mon 2020-05-18 21:34:14 UTC] ; stop_time=[n/a] ; pid=1105 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPostEx": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", 2020-05-18T21:35:52.774 INFO:teuthology.task.ansible.out:"GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-05-18 21:34:14 UTC", "InactiveExitTimestampMonotonic": "16646644", "InvocationID": "d2cac275038241c9991263c647b431a7", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127783", "LimitNPROCSoft": "127783", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127783", "LimitSIGPENDINGSoft": "127783", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "1105", "MemoryAccounting": "yes", "MemoryCurrent": "942080", "Memo 2020-05-18T21:35:52.776 INFO:teuthology.task.ansible.out:ryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/run/nagios/nrpe.pid", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-05-18 21:34:14 UTC", "StateChangeTimestampMonotonic": "16646644", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": " 2020-05-18T21:35:52.777 INFO:teuthology.task.ansible.out:no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "38334", "TimeoutAbortUSec": "1min", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "110", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-18T21:35:52.811 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:52 +0000 (0:00:00.052) 0:01:08.860 ************ 2020-05-18T21:35:52.821 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-05-18T21:35:52.822 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:52.859 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:52 +0000 (0:00:00.047) 0:01:08.907 ************ 2020-05-18T21:35:52.869 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:52.913 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:52 +0000 (0:00:00.054) 0:01:08.962 ************ 2020-05-18T21:35:52.921 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-05-18T21:35:52.923 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:52.964 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:52 +0000 (0:00:00.051) 0:01:09.013 ************ 2020-05-18T21:35:52.973 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.020 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.055) 0:01:09.068 ************ 2020-05-18T21:35:53.028 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-05-18T21:35:53.030 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.067 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.047) 0:01:09.116 ************ 2020-05-18T21:35:53.076 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.119 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.052) 0:01:09.168 ************ 2020-05-18T21:35:53.128 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-05-18T21:35:53.130 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.167 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.047) 0:01:09.215 ************ 2020-05-18T21:35:53.175 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.213 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.046) 0:01:09.262 ************ 2020-05-18T21:35:53.221 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-05-18T21:35:53.223 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.259 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.046) 0:01:09.308 ************ 2020-05-18T21:35:53.268 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.315 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.055) 0:01:09.363 ************ 2020-05-18T21:35:53.325 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-05-18T21:35:53.327 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.365 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.050) 0:01:09.414 ************ 2020-05-18T21:35:53.374 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.420 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.051) 0:01:09.465 ************ 2020-05-18T21:35:53.426 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-05-18T21:35:53.428 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.467 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.051) 0:01:09.516 ************ 2020-05-18T21:35:53.476 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.515 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.047) 0:01:09.564 ************ 2020-05-18T21:35:53.524 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-05-18T21:35:53.526 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.567 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.051) 0:01:09.616 ************ 2020-05-18T21:35:53.576 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.617 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.049) 0:01:09.665 ************ 2020-05-18T21:35:53.626 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-05-18T21:35:53.628 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.667 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.050) 0:01:09.716 ************ 2020-05-18T21:35:53.677 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.724 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.056) 0:01:09.773 ************ 2020-05-18T21:35:53.733 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-05-18T21:35:53.735 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.773 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.049) 0:01:09.822 ************ 2020-05-18T21:35:53.781 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:53.821 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.046) 0:01:09.869 ************ 2020-05-18T21:35:53.828 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-05-18T21:35:53.830 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-05-18T21:35:53.868 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.048) 0:01:09.917 ************ 2020-05-18T21:35:53.876 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-05-18T21:35:53.923 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.054) 0:01:09.971 ************ 2020-05-18T21:35:53.933 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-05-18T21:35:53.935 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-05-18T21:35:53.976 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:53 +0000 (0:00:00.053) 0:01:10.025 ************ 2020-05-18T21:35:53.983 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-05-18T21:35:54.024 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.048) 0:01:10.073 ************ 2020-05-18T21:35:54.032 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-05-18T21:35:54.034 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose", "python3-virtualenv", "python3-configobj", "python3-gevent", "python3-numpy", "python3-matplotlib", "python3-setuptools"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"} 2020-05-18T21:35:54.081 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.056) 0:01:10.130 ************ 2020-05-18T21:35:54.090 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose", "python3-virtualenv", "python3-configobj", "python3-gevent", "python3-numpy", "python3-matplotlib", "python3-setuptools"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"} 2020-05-18T21:35:54.134 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.052) 0:01:10.183 ************ 2020-05-18T21:35:54.142 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-05-18T21:35:54.144 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-05-18T21:35:54.182 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.047) 0:01:10.230 ************ 2020-05-18T21:35:54.190 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-05-18T21:35:54.229 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.046) 0:01:10.277 ************ 2020-05-18T21:35:54.319 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-05-18T21:35:54.321 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-05-18T21:35:54.357 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.128) 0:01:10.406 ************ 2020-05-18T21:35:54.367 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-05-18T21:35:54.403 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.046) 0:01:10.452 ************ 2020-05-18T21:35:54.503 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-05-18T21:35:54.505 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "gid": 1214, "name": "ubuntu", "state": "present", "system": false} 2020-05-18T21:35:54.548 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.145) 0:01:10.597 ************ 2020-05-18T21:35:54.555 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "gid": 1214, "name": "ubuntu", "state": "present", "system": false} 2020-05-18T21:35:54.593 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.044) 0:01:10.641 ************ 2020-05-18T21:35:54.601 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-05-18T21:35:54.603 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:54.641 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.048) 0:01:10.690 ************ 2020-05-18T21:35:54.651 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:54.690 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.048) 0:01:10.739 ************ 2020-05-18T21:35:54.699 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-05-18T21:35:54.701 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:54.745 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.054) 0:01:10.794 ************ 2020-05-18T21:35:54.754 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:54.793 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.047) 0:01:10.841 ************ 2020-05-18T21:35:54.911 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-05-18T21:35:54.913 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-05-18T21:35:54.963 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:54 +0000 (0:00:00.170) 0:01:11.012 ************ 2020-05-18T21:35:54.975 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-05-18T21:35:55.015 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:55 +0000 (0:00:00.051) 0:01:11.063 ************ 2020-05-18T21:35:55.436 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-05-18T21:35:55.438 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-05-18T21:35:55.475 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:55 +0000 (0:00:00.460) 0:01:11.524 ************ 2020-05-18T21:35:55.513 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:55 +0000 (0:00:00.038) 0:01:11.562 ************ 2020-05-18T21:35:55.522 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-05-18T21:35:55.524 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi080"}, "changed": false} 2020-05-18T21:35:55.573 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:55 +0000 (0:00:00.060) 0:01:11.622 ************ 2020-05-18T21:35:55.584 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi196"}, "changed": false} 2020-05-18T21:35:55.624 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:55 +0000 (0:00:00.051) 0:01:11.673 ************ 2020-05-18T21:35:56.672 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-05-18T21:35:56.674 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi080.front.sepia.ceph.com", "ansible_hostname": "smithi080", "ansible_nodename": "smithi080"}, "changed": true, "name": "smithi080"} 2020-05-18T21:35:56.708 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:56 +0000 (0:00:01.083) 0:01:12.757 ************ 2020-05-18T21:35:56.717 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi196.front.sepia.ceph.com", "ansible_hostname": "smithi196", "ansible_nodename": "smithi196"}, "changed": true, "name": "smithi196"} 2020-05-18T21:35:56.755 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:56 +0000 (0:00:00.047) 0:01:12.804 ************ 2020-05-18T21:35:57.060 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-05-18T21:35:57.062 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-05-18T21:35:57.112 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:57 +0000 (0:00:00.356) 0:01:13.160 ************ 2020-05-18T21:35:57.135 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-05-18T21:35:57.173 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:57 +0000 (0:00:00.061) 0:01:13.222 ************ 2020-05-18T21:35:57.505 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-05-18T21:35:57.507 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-05-18T21:35:57.544 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:57 +0000 (0:00:00.371) 0:01:13.593 ************ 2020-05-18T21:35:57.564 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-05-18T21:35:57.611 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:57 +0000 (0:00:00.066) 0:01:13.660 ************ 2020-05-18T21:35:58.126 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-05-18T21:35:58.128 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-05-18T21:35:58.163 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.551) 0:01:14.212 ************ 2020-05-18T21:35:58.182 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-05-18T21:35:58.219 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.055) 0:01:14.268 ************ 2020-05-18T21:35:58.227 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-05-18T21:35:58.229 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.264 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.045) 0:01:14.313 ************ 2020-05-18T21:35:58.272 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.309 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.045) 0:01:14.358 ************ 2020-05-18T21:35:58.317 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-05-18T21:35:58.319 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.355 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.045) 0:01:14.403 ************ 2020-05-18T21:35:58.362 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.398 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.043) 0:01:14.447 ************ 2020-05-18T21:35:58.415 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.460 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.061) 0:01:14.509 ************ 2020-05-18T21:35:58.472 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.516 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.054) 0:01:14.564 ************ 2020-05-18T21:35:58.524 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-05-18T21:35:58.526 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.568 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.052) 0:01:14.617 ************ 2020-05-18T21:35:58.579 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.626 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.057) 0:01:14.675 ************ 2020-05-18T21:35:58.635 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-05-18T21:35:58.636 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.678 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.051) 0:01:14.727 ************ 2020-05-18T21:35:58.686 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.723 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.045) 0:01:14.772 ************ 2020-05-18T21:35:58.731 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-05-18T21:35:58.732 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.768 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.045) 0:01:14.817 ************ 2020-05-18T21:35:58.779 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.818 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.050) 0:01:14.867 ************ 2020-05-18T21:35:58.827 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-05-18T21:35:58.827 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.864 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.046) 0:01:14.913 ************ 2020-05-18T21:35:58.873 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.911 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.046) 0:01:14.960 ************ 2020-05-18T21:35:58.919 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-05-18T21:35:58.922 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:58.958 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.046) 0:01:15.007 ************ 2020-05-18T21:35:58.969 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.006 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:58 +0000 (0:00:00.047) 0:01:15.054 ************ 2020-05-18T21:35:59.015 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-05-18T21:35:59.053 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.047) 0:01:15.102 ************ 2020-05-18T21:35:59.100 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.046) 0:01:15.148 ************ 2020-05-18T21:35:59.108 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-05-18T21:35:59.145 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.045) 0:01:15.194 ************ 2020-05-18T21:35:59.192 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.046) 0:01:15.241 ************ 2020-05-18T21:35:59.200 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-05-18T21:35:59.244 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.052) 0:01:15.293 ************ 2020-05-18T21:35:59.291 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.046) 0:01:15.340 ************ 2020-05-18T21:35:59.300 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-05-18T21:35:59.302 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.337 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.045) 0:01:15.386 ************ 2020-05-18T21:35:59.348 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.389 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.051) 0:01:15.438 ************ 2020-05-18T21:35:59.397 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-05-18T21:35:59.398 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.438 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.048) 0:01:15.486 ************ 2020-05-18T21:35:59.446 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.448 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.486 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.048) 0:01:15.535 ************ 2020-05-18T21:35:59.494 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-05-18T21:35:59.496 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.533 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.047) 0:01:15.582 ************ 2020-05-18T21:35:59.542 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.582 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.048) 0:01:15.630 ************ 2020-05-18T21:35:59.590 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-18T21:35:59.591 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.628 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.046) 0:01:15.677 ************ 2020-05-18T21:35:59.642 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.681 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.052) 0:01:15.730 ************ 2020-05-18T21:35:59.689 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-05-18T21:35:59.691 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.727 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.045) 0:01:15.775 ************ 2020-05-18T21:35:59.734 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.773 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.046) 0:01:15.822 ************ 2020-05-18T21:35:59.783 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-05-18T21:35:59.785 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.836 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.062) 0:01:15.885 ************ 2020-05-18T21:35:59.844 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.884 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.048) 0:01:15.933 ************ 2020-05-18T21:35:59.892 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-05-18T21:35:59.894 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.934 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.050) 0:01:15.983 ************ 2020-05-18T21:35:59.943 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:35:59.982 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:35:59 +0000 (0:00:00.047) 0:01:16.031 ************ 2020-05-18T21:35:59.989 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-05-18T21:35:59.991 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.029 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.046) 0:01:16.078 ************ 2020-05-18T21:36:00.038 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.078 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.048) 0:01:16.127 ************ 2020-05-18T21:36:00.086 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-05-18T21:36:00.086 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.124 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.046) 0:01:16.173 ************ 2020-05-18T21:36:00.132 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.170 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.045) 0:01:16.219 ************ 2020-05-18T21:36:00.181 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.219 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.048) 0:01:16.267 ************ 2020-05-18T21:36:00.227 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.265 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.046) 0:01:16.314 ************ 2020-05-18T21:36:00.273 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-05-18T21:36:00.274 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.319 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.053) 0:01:16.368 ************ 2020-05-18T21:36:00.328 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.366 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.046) 0:01:16.415 ************ 2020-05-18T21:36:00.372 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-05-18T21:36:00.374 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.410 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.043) 0:01:16.458 ************ 2020-05-18T21:36:00.417 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.454 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.044) 0:01:16.503 ************ 2020-05-18T21:36:00.461 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-05-18T21:36:00.463 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.500 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.045) 0:01:16.549 ************ 2020-05-18T21:36:00.507 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.545 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.045) 0:01:16.594 ************ 2020-05-18T21:36:00.551 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-05-18T21:36:00.553 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.590 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.044) 0:01:16.639 ************ 2020-05-18T21:36:00.597 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.635 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.045) 0:01:16.684 ************ 2020-05-18T21:36:00.642 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-05-18T21:36:00.644 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.681 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.045) 0:01:16.729 ************ 2020-05-18T21:36:00.689 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.730 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.048) 0:01:16.778 ************ 2020-05-18T21:36:00.736 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-05-18T21:36:00.738 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.774 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.044) 0:01:16.823 ************ 2020-05-18T21:36:00.783 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.821 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.046) 0:01:16.870 ************ 2020-05-18T21:36:00.828 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-05-18T21:36:00.830 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.866 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.045) 0:01:16.915 ************ 2020-05-18T21:36:00.874 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:00.912 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:00 +0000 (0:00:00.046) 0:01:16.961 ************ 2020-05-18T21:36:01.012 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-05-18T21:36:01.014 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005241", "end": "2020-05-18 21:36:00.603253", "rc": 0, "start": "2020-05-18 21:36:00.598012", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:36:01.053 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:01 +0000 (0:00:00.140) 0:01:17.102 ************ 2020-05-18T21:36:01.073 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005410", "end": "2020-05-18 21:36:01.487217", "rc": 0, "start": "2020-05-18 21:36:01.481807", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:36:01.114 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:01 +0000 (0:00:00.061) 0:01:17.163 ************ 2020-05-18T21:36:01.126 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-05-18T21:36:01.171 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:01 +0000 (0:00:00.056) 0:01:17.220 ************ 2020-05-18T21:36:01.220 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:01 +0000 (0:00:00.048) 0:01:17.268 ************ 2020-05-18T21:36:01.524 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-05-18T21:36:01.525 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-05-18T21:36:01.561 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:01 +0000 (0:00:00.341) 0:01:17.610 ************ 2020-05-18T21:36:01.582 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-05-18T21:36:01.618 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:01 +0000 (0:00:00.056) 0:01:17.667 ************ 2020-05-18T21:36:01.626 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-05-18T21:36:01.628 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:01.665 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:01 +0000 (0:00:00.046) 0:01:17.713 ************ 2020-05-18T21:36:01.673 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:01.710 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:01 +0000 (0:00:00.045) 0:01:17.759 ************ 2020-05-18T21:36:03.139 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-05-18T21:36:03.140 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-05-18T21:36:03.142 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-05-18T21:36:04.241 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-05-18T21:36:04.280 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:04 +0000 (0:00:02.569) 0:01:20.329 ************ 2020-05-18T21:36:04.290 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-05-18T21:36:04.330 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:04 +0000 (0:00:00.050) 0:01:20.379 ************ 2020-05-18T21:36:04.899 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-05-18T21:36:04.901 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"cache_update_time": 1589837434, "cache_updated": false, "changed": false} 2020-05-18T21:36:04.938 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:04 +0000 (0:00:00.608) 0:01:20.987 ************ 2020-05-18T21:36:04.958 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"cache_update_time": 1589837492, "cache_updated": false, "changed": false} 2020-05-18T21:36:04.993 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:04 +0000 (0:00:00.055) 0:01:21.042 ************ 2020-05-18T21:36:05.002 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-05-18T21:36:05.040 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:05 +0000 (0:00:00.046) 0:01:21.089 ************ 2020-05-18T21:36:05.088 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:05 +0000 (0:00:00.047) 0:01:21.136 ************ 2020-05-18T21:36:05.228 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-05-18T21:36:05.230 INFO:teuthology.task.ansible.out: [WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [smithi196.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.032854", "end": "2020-05-18 21:36:05.627612", "rc": 0, "start": "2020-05-18 21:36:05.594758", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:36:05.266 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:05 +0000 (0:00:00.178) 0:01:21.315 ************ 2020-05-18T21:36:05.274 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.033771", "end": "2020-05-18 21:36:04.857894", "rc": 0, "start": "2020-05-18 21:36:04.824123", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:36:05.316 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:05 +0000 (0:00:00.049) 0:01:21.365 ************ 2020-05-18T21:36:07.344 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-05-18T21:36:07.346 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"cache_update_time": 1589837434, "cache_updated": false, "changed": false} 2020-05-18T21:36:07.402 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:07 +0000 (0:00:02.085) 0:01:23.451 ************ 2020-05-18T21:36:07.414 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"cache_update_time": 1589837492, "cache_updated": false, "changed": false} 2020-05-18T21:36:07.452 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:07 +0000 (0:00:00.049) 0:01:23.501 ************ 2020-05-18T21:36:19.681 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-05-18T21:36:19.684 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker ipxe-qemu ipxe-qemu-256k-compat-efi-roms\n libboost-iostreams1.71.0 libbrlapi0.7 libcacard0 libiscsi7 libpmem1\n libslirp0 libspice-server1 libusbredirparser1 libvirglrenderer1 msr-tools\n ovmf qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 48 not upgraded.\nAfter this operation, 62.8 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 149536 files and directories currently installed.)\r\nRemoving qemu-utils (1:4.2-3ubuntu6) ...\r\nRemoving qemu-system-x86 (1:4.2-3ubuntu6) ...\r\nRemoving qemu-system-common (1:4.2-3ubuntu6) ...\r\nRemoving qemu-block-extra:amd64 (1:4.2-3ubuntu6) ...\r\nRemoving librbd1 (15.2.1-0ubuntu1) ...\r\nRemoving librados2 (15.2.1-0ubuntu1) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for l 2020-05-18T21:36:19.685 INFO:teuthology.task.ansible.out:ibc-bin (2.31-0ubuntu9) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker ipxe-qemu ipxe-qemu-256k-compat-efi-roms", " libboost-iostreams1.71.0 libbrlapi0.7 libcacard0 libiscsi7 libpmem1", " libslirp0 libspice-server1 libusbredirparser1 libvirglrenderer1 msr-tools", " ovmf qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 48 not upgraded.", "After this operation, 62.8 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 149536 files and directories currently installed.)", "Removing qemu-utils (1:4.2-3ubuntu6) ...", "Removing qemu-system-x86 (1:4.2-3ubuntu6) ...", "Removing qemu-system-common (1:4.2-3ubuntu6) ...", "Removing qemu-block-extra:amd64 (1:4.2-3ubuntu6) ...", "Removing librbd1 (15.2.1-0ubuntu1) ...", "Removing librados2 (15.2.1-0ubuntu1) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9) ..."]} 2020-05-18T21:36:19.720 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:19 +0000 (0:00:12.268) 0:01:35.769 ************ 2020-05-18T21:36:19.734 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker ipxe-qemu ipxe-qemu-256k-compat-efi-roms\n libboost-iostreams1.71.0 libbrlapi0.7 libcacard0 libiscsi7 libpmem1\n libslirp0 libspice-server1 libusbredirparser1 libvirglrenderer1 msr-tools\n ovmf qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 48 not upgraded.\nAfter this operation, 62.8 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 149536 files and directories currently installed.)\r\nRemoving qemu-utils (1:4.2-3ubuntu6) ...\r\nRemoving qemu-system-x86 (1:4.2-3ubuntu6) ...\r\nRemoving qemu-system-common (1:4.2-3ubuntu6) ...\r\nRemoving qemu-block-extra:amd64 (1:4.2-3ubuntu6) ...\r\nRemoving librbd1 (15.2.1-0ubuntu1) ...\r\nRemoving librados2 (15.2.1-0ubuntu1) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for l 2020-05-18T21:36:19.736 INFO:teuthology.task.ansible.out:ibc-bin (2.31-0ubuntu9) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker ipxe-qemu ipxe-qemu-256k-compat-efi-roms", " libboost-iostreams1.71.0 libbrlapi0.7 libcacard0 libiscsi7 libpmem1", " libslirp0 libspice-server1 libusbredirparser1 libvirglrenderer1 msr-tools", " ovmf qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 48 not upgraded.", "After this operation, 62.8 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 149536 files and directories currently installed.)", "Removing qemu-utils (1:4.2-3ubuntu6) ...", "Removing qemu-system-x86 (1:4.2-3ubuntu6) ...", "Removing qemu-system-common (1:4.2-3ubuntu6) ...", "Removing qemu-block-extra:amd64 (1:4.2-3ubuntu6) ...", "Removing librbd1 (15.2.1-0ubuntu1) ...", "Removing librados2 (15.2.1-0ubuntu1) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9) ..."]} 2020-05-18T21:36:19.774 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:19 +0000 (0:00:00.054) 0:01:35.823 ************ 2020-05-18T21:36:31.770 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-05-18T21:36:31.772 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"cache_update_time": 1589837434, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker ipxe-qemu ipxe-qemu-256k-compat-efi-roms\n libboost-iostreams1.71.0 libbrlapi0.7 libcacard0 libiscsi7 libpmem1\n libslirp0 libspice-server1 libusbredirparser1 libvirglrenderer1 msr-tools\n ovmf qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apt-utils libapt-pkg6.0\nSuggested packages:\n apt-doc aptitude | synaptic | wajig\nThe following packages will be upgraded:\n apt apt-utils libapt-pkg6.0\n3 upgraded, 0 newly installed, 0 to remove and 45 not upgraded.\nNeed to get 2336 kB of archives.\nAfter this operation, 6144 B disk space will be freed.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapt-pkg6.0 amd64 2.0.2ubuntu0.1 [832 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt amd64 2.0.2ubuntu0.1 [1290 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt-utils amd64 2.0.2ubuntu0.1 [214 kB]\nFetched 2336 kB in 1s (2680 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database .. 2020-05-18T21:36:31.774 INFO:teuthology.task.ansible.out:. 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 149464 files and directories currently installed.)\r\nPreparing to unpack .../libapt-pkg6.0_2.0.2ubuntu0.1_amd64.deb ...\r\nUnpacking libapt-pkg6.0:amd64 (2.0.2ubuntu0.1) over (2.0.2) ...\r\nSetting up libapt-pkg6.0:amd64 (2.0.2ubuntu0.1) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 149464 files and directories currently installed.)\r\nPreparing to unpack .../apt_2.0.2ubuntu0.1_amd64.deb ...\r\nUnpacking apt (2.0.2ubuntu0.1) over (2.0.2) ...\r\nSetting up apt (2.0.2ubuntu0.1) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 149464 files and directories currently installed.)\r\nPreparing to unpack .../apt-utils_2.0.2ubuntu0.1_amd64.deb ...\r\nUnpacking apt-utils (2.0.2ubuntu0.1) over (2.0.2) ...\r\nSetting up apt-utils (2.0.2ubuntu0.1) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\ 2020-05-18T21:36:31.776 INFO:teuthology.task.ansible.out:r\nProcessing triggers for libc-bin (2.31-0ubuntu9) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker ipxe-qemu ipxe-qemu-256k-compat-efi-roms", " libboost-iostreams1.71.0 libbrlapi0.7 libcacard0 libiscsi7 libpmem1", " libslirp0 libspice-server1 libusbredirparser1 libvirglrenderer1 msr-tools", " ovmf qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apt-utils libapt-pkg6.0", "Suggested packages:", " apt-doc aptitude | synaptic | wajig", "The following packages will be upgraded:", " apt apt-utils libapt-pkg6.0", "3 upgraded, 0 newly installed, 0 to remove and 45 not upgraded.", "Need to get 2336 kB of archives.", "After this operation, 6144 B disk space will be freed.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapt-pkg6.0 amd64 2.0.2ubuntu0.1 [832 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt amd64 2.0.2ubuntu0.1 [1290 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt-utils amd64 2.0.2ubuntu0.1 [214 kB]", "Fetched 2336 kB in 1s (2680 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 149464 files and directories currently installed.)", "Prep 2020-05-18T21:36:31.777 INFO:teuthology.task.ansible.out:aring to unpack .../libapt-pkg6.0_2.0.2ubuntu0.1_amd64.deb ...", "Unpacking libapt-pkg6.0:amd64 (2.0.2ubuntu0.1) over (2.0.2) ...", "Setting up libapt-pkg6.0:amd64 (2.0.2ubuntu0.1) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 149464 files and directories currently installed.)", "Preparing to unpack .../apt_2.0.2ubuntu0.1_amd64.deb ...", "Unpacking apt (2.0.2ubuntu0.1) over (2.0.2) ...", "Setting up apt (2.0.2ubuntu0.1) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 149464 files and directories currently installed.)", "Preparing to unpack .../apt-utils_2.0.2ubuntu0.1_amd64.deb ...", "Unpacking apt-utils (2.0.2ubuntu0.1) over (2.0.2) ...", "Setting up apt-utils (2.0.2ubuntu0.1) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9) ..."]} 2020-05-18T21:36:31.824 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:31 +0000 (0:00:12.049) 0:01:47.873 ************ 2020-05-18T21:36:31.835 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"cache_update_time": 1589837492, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker ipxe-qemu ipxe-qemu-256k-compat-efi-roms\n libboost-iostreams1.71.0 libbrlapi0.7 libcacard0 libiscsi7 libpmem1\n libslirp0 libspice-server1 libusbredirparser1 libvirglrenderer1 msr-tools\n ovmf qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apt-utils libapt-pkg6.0\nSuggested packages:\n apt-doc aptitude | synaptic | wajig\nThe following packages will be upgraded:\n apt apt-utils libapt-pkg6.0\n3 upgraded, 0 newly installed, 0 to remove and 45 not upgraded.\nNeed to get 2336 kB of archives.\nAfter this operation, 6144 B disk space will be freed.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapt-pkg6.0 amd64 2.0.2ubuntu0.1 [832 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt amd64 2.0.2ubuntu0.1 [1290 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt-utils amd64 2.0.2ubuntu0.1 [214 kB]\nFetched 2336 kB in 1s (2684 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database .. 2020-05-18T21:36:31.836 INFO:teuthology.task.ansible.out:. 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 149464 files and directories currently installed.)\r\nPreparing to unpack .../libapt-pkg6.0_2.0.2ubuntu0.1_amd64.deb ...\r\nUnpacking libapt-pkg6.0:amd64 (2.0.2ubuntu0.1) over (2.0.2) ...\r\nSetting up libapt-pkg6.0:amd64 (2.0.2ubuntu0.1) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 149464 files and directories currently installed.)\r\nPreparing to unpack .../apt_2.0.2ubuntu0.1_amd64.deb ...\r\nUnpacking apt (2.0.2ubuntu0.1) over (2.0.2) ...\r\nSetting up apt (2.0.2ubuntu0.1) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 149464 files and directories currently installed.)\r\nPreparing to unpack .../apt-utils_2.0.2ubuntu0.1_amd64.deb ...\r\nUnpacking apt-utils (2.0.2ubuntu0.1) over (2.0.2) ...\r\nSetting up apt-utils (2.0.2ubuntu0.1) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\ 2020-05-18T21:36:31.838 INFO:teuthology.task.ansible.out:r\nProcessing triggers for libc-bin (2.31-0ubuntu9) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker ipxe-qemu ipxe-qemu-256k-compat-efi-roms", " libboost-iostreams1.71.0 libbrlapi0.7 libcacard0 libiscsi7 libpmem1", " libslirp0 libspice-server1 libusbredirparser1 libvirglrenderer1 msr-tools", " ovmf qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apt-utils libapt-pkg6.0", "Suggested packages:", " apt-doc aptitude | synaptic | wajig", "The following packages will be upgraded:", " apt apt-utils libapt-pkg6.0", "3 upgraded, 0 newly installed, 0 to remove and 45 not upgraded.", "Need to get 2336 kB of archives.", "After this operation, 6144 B disk space will be freed.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapt-pkg6.0 amd64 2.0.2ubuntu0.1 [832 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt amd64 2.0.2ubuntu0.1 [1290 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt-utils amd64 2.0.2ubuntu0.1 [214 kB]", "Fetched 2336 kB in 1s (2684 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 149464 files and directories currently installed.)", "Prep 2020-05-18T21:36:31.840 INFO:teuthology.task.ansible.out:aring to unpack .../libapt-pkg6.0_2.0.2ubuntu0.1_amd64.deb ...", "Unpacking libapt-pkg6.0:amd64 (2.0.2ubuntu0.1) over (2.0.2) ...", "Setting up libapt-pkg6.0:amd64 (2.0.2ubuntu0.1) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 149464 files and directories currently installed.)", "Preparing to unpack .../apt_2.0.2ubuntu0.1_amd64.deb ...", "Unpacking apt (2.0.2ubuntu0.1) over (2.0.2) ...", "Setting up apt (2.0.2ubuntu0.1) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 149464 files and directories currently installed.)", "Preparing to unpack .../apt-utils_2.0.2ubuntu0.1_amd64.deb ...", "Unpacking apt-utils (2.0.2ubuntu0.1) over (2.0.2) ...", "Setting up apt-utils (2.0.2ubuntu0.1) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9) ..."]} 2020-05-18T21:36:31.874 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:31 +0000 (0:00:00.049) 0:01:47.923 ************ 2020-05-18T21:36:31.883 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-05-18T21:36:31.885 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:31.934 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:31 +0000 (0:00:00.054) 0:01:47.978 ************ 2020-05-18T21:36:31.939 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:31.980 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:31 +0000 (0:00:00.051) 0:01:48.029 ************ 2020-05-18T21:36:41.306 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-05-18T21:36:41.308 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"cache_update_time": 1589837434, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.25.1-1ubuntu3).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 45 not upgraded.\nNeed to get 13.7 MB of archives.\nAfter this operation, 62.8 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.1-0ubuntu2 [3236 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.1-0ubuntu2 [1611 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6 [56.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6 [1054 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6 [6741 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-utils amd64 1:4.2-3ubuntu6 [965 kB]\nFetched 13.7 MB in 1s (11.5 MB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database 2020-05-18T21:36:41.309 INFO:teuthology.task.ansible.out: ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 149464 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_15.2.1-0ubuntu2_amd64.deb ...\r\nUnpacking librados2 (15.2.1-0ubuntu2) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_15.2.1-0ubuntu2_amd64.deb ...\r\nUnpacking librbd1 (15.2.1-0ubuntu2) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a4.2-3ubuntu6_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a4.2-3ubuntu6_amd64.deb ...\r\nUnpacking qemu-system-common (1:4.2-3ubuntu6) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a4.2-3ubuntu6_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:4.2-3ubuntu6) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a4.2-3ubuntu6_amd64.deb ...\r\nUnpacking qemu-utils (1:4.2-3ubuntu6) ...\r\nSetting up librados2 (15.2.1-0ubuntu2) ...\r\nSetting up librbd1 (15.2.1-0ubuntu2) ...\r\nSetting up qemu-block-extra:amd64 (1:4.2-3ubuntu6) ...\r\nSetting up qemu-system-common (1:4.2-3ubuntu6) ...\r\nSetting up qemu-system-x86 (1:4.2-3ubuntu6) ...\r\nSetting up qemu-utils (1:4.2-3ubuntu6) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.25.1-1ubuntu3).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qe 2020-05-18T21:36:41.311 INFO:teuthology.task.ansible.out:mu-system-common qemu-utils", "Suggested packages:", " samba vde2 debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 45 not upgraded.", "Need to get 13.7 MB of archives.", "After this operation, 62.8 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.1-0ubuntu2 [3236 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.1-0ubuntu2 [1611 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6 [56.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6 [1054 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6 [6741 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-utils amd64 1:4.2-3ubuntu6 [965 kB]", "Fetched 13.7 MB in 1s (11.5 MB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 149464 files and directories currently installed.)", "Preparing to unpack .../0-librados2_15.2.1-0ubuntu2_amd64.deb ...", "Unpacking librados2 (15.2.1-0ubuntu2) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1- 2020-05-18T21:36:41.313 INFO:teuthology.task.ansible.out:librbd1_15.2.1-0ubuntu2_amd64.deb ...", "Unpacking librbd1 (15.2.1-0ubuntu2) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a4.2-3ubuntu6_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a4.2-3ubuntu6_amd64.deb ...", "Unpacking qemu-system-common (1:4.2-3ubuntu6) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a4.2-3ubuntu6_amd64.deb ...", "Unpacking qemu-system-x86 (1:4.2-3ubuntu6) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a4.2-3ubuntu6_amd64.deb ...", "Unpacking qemu-utils (1:4.2-3ubuntu6) ...", "Setting up librados2 (15.2.1-0ubuntu2) ...", "Setting up librbd1 (15.2.1-0ubuntu2) ...", "Setting up qemu-block-extra:amd64 (1:4.2-3ubuntu6) ...", "Setting up qemu-system-common (1:4.2-3ubuntu6) ...", "Setting up qemu-system-x86 (1:4.2-3ubuntu6) ...", "Setting up qemu-utils (1:4.2-3ubuntu6) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9) ..."]} 2020-05-18T21:36:41.351 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:41 +0000 (0:00:09.370) 0:01:57.400 ************ 2020-05-18T21:36:41.416 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"cache_update_time": 1589837492, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.25.1-1ubuntu3).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 45 not upgraded.\nNeed to get 13.7 MB of archives.\nAfter this operation, 62.8 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.1-0ubuntu2 [3236 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.1-0ubuntu2 [1611 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6 [56.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6 [1054 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6 [6741 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-utils amd64 1:4.2-3ubuntu6 [965 kB]\nFetched 13.7 MB in 1s (11.5 MB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database 2020-05-18T21:36:41.418 INFO:teuthology.task.ansible.out: ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 149464 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_15.2.1-0ubuntu2_amd64.deb ...\r\nUnpacking librados2 (15.2.1-0ubuntu2) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_15.2.1-0ubuntu2_amd64.deb ...\r\nUnpacking librbd1 (15.2.1-0ubuntu2) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a4.2-3ubuntu6_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a4.2-3ubuntu6_amd64.deb ...\r\nUnpacking qemu-system-common (1:4.2-3ubuntu6) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a4.2-3ubuntu6_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:4.2-3ubuntu6) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a4.2-3ubuntu6_amd64.deb ...\r\nUnpacking qemu-utils (1:4.2-3ubuntu6) ...\r\nSetting up librados2 (15.2.1-0ubuntu2) ...\r\nSetting up librbd1 (15.2.1-0ubuntu2) ...\r\nSetting up qemu-block-extra:amd64 (1:4.2-3ubuntu6) ...\r\nSetting up qemu-system-common (1:4.2-3ubuntu6) ...\r\nSetting up qemu-system-x86 (1:4.2-3ubuntu6) ...\r\nSetting up qemu-utils (1:4.2-3ubuntu6) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.25.1-1ubuntu3).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qe 2020-05-18T21:36:41.420 INFO:teuthology.task.ansible.out:mu-system-common qemu-utils", "Suggested packages:", " samba vde2 debootstrap", "The following 2020-05-18T21:36:41.426 INFO:teuthology.task.ansible.out:NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 45 not upgraded.", "Need to get 13.7 MB of archives.", "After this operation, 62.8 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.1-0ubuntu2 [3236 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.1-0ubuntu2 [1611 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6 [56.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6 [1054 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6 [6741 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 qemu-utils amd64 1:4.2-3ubuntu6 [965 kB]", "Fetched 13.7 MB in 1s (11.5 MB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 149464 files and directories currently installed.)", "Preparing to unpack .../0-librados2_15.2.1-0ubuntu2_amd64.deb ...", "Unpacking librados2 (15.2.1-0ubuntu2) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_15.2.1-0ubuntu2_amd64.deb ...", "Unpacking librbd1 (15.2.1-0ubuntu2) ...", "Selecting pr 2020-05-18T21:36:41.428 INFO:teuthology.task.ansible.out:eviously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a4.2-3ubuntu6_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a4.2-3ubuntu6_amd64.deb ...", "Unpacking qemu-system-common (1:4.2-3ubuntu6) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a4.2-3ubuntu6_amd64.deb ...", "Unpacking qemu-system-x86 (1:4.2-3ubuntu6) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a4.2-3ubuntu6_amd64.deb ...", "Unpacking qemu-utils (1:4.2-3ubuntu6) ...", "Setting up librados2 (15.2.1-0ubuntu2) ...", "Setting up librbd1 (15.2.1-0ubuntu2) ...", "Setting up qemu-block-extra:amd64 (1:4.2-3ubuntu6) ...", "Setting up qemu-system-common (1:4.2-3ubuntu6) ...", "Setting up qemu-system-x86 (1:4.2-3ubuntu6) ...", "Setting up qemu-utils (1:4.2-3ubuntu6) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9) ..."]} 2020-05-18T21:36:41.468 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:41 +0000 (0:00:00.116) 0:01:57.517 ************ 2020-05-18T21:36:41.976 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-05-18T21:36:41.978 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"cache_update_time": 1589837434, "cache_updated": false, "changed": false} 2020-05-18T21:36:42.017 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:42 +0000 (0:00:00.548) 0:01:58.066 ************ 2020-05-18T21:36:42.101 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"cache_update_time": 1589837492, "cache_updated": false, "changed": false} 2020-05-18T21:36:42.144 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:42 +0000 (0:00:00.127) 0:01:58.193 ************ 2020-05-18T21:36:42.164 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-05-18T21:36:42.166 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-05-18T21:36:42.206 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:42 +0000 (0:00:00.061) 0:01:58.255 ************ 2020-05-18T21:36:42.247 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-05-18T21:36:42.249 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:42.285 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:42 +0000 (0:00:00.077) 0:01:58.333 ************ 2020-05-18T21:36:42.297 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-05-18T21:36:42.299 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-05-18T21:36:42.340 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:42 +0000 (0:00:00.053) 0:01:58.388 ************ 2020-05-18T21:36:42.385 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-05-18T21:36:42.387 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:42.421 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:42 +0000 (0:00:00.081) 0:01:58.470 ************ 2020-05-18T21:36:42.434 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-05-18T21:36:42.470 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:42 +0000 (0:00:00.048) 0:01:58.519 ************ 2020-05-18T21:36:42.570 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-05-18T21:36:42.607 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:42 +0000 (0:00:00.137) 0:01:58.656 ************ 2020-05-18T21:36:42.843 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-05-18T21:36:42.845 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-05-18T21:36:42.881 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:42 +0000 (0:00:00.273) 0:01:58.929 ************ 2020-05-18T21:36:42.995 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-05-18T21:36:43.031 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:43 +0000 (0:00:00.150) 0:01:59.080 ************ 2020-05-18T21:36:43.071 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-05-18T21:36:43.072 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1214, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-05-18T21:36:43.109 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:43 +0000 (0:00:00.077) 0:01:59.158 ************ 2020-05-18T21:36:43.203 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1214, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-05-18T21:36:43.241 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:43 +0000 (0:00:00.131) 0:01:59.290 ************ 2020-05-18T21:36:43.271 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-05-18T21:36:43.272 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-18T21:36:43.311 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:43 +0000 (0:00:00.070) 0:01:59.360 ************ 2020-05-18T21:36:43.398 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-18T21:36:43.440 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:43 +0000 (0:00:00.127) 0:01:59.488 ************ 2020-05-18T21:36:43.590 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-05-18T21:36:43.592 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Mon 2020-05-18 21:34:08 UTC", "ActiveEnterTimestampMonotonic": "17399936", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target system.slice -.mount basic.target sysinit.target network.target systemd-tmpfiles-setup.service systemd-journald.socket remote-fs.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-05-18 21:34:07 UTC", "AssertTimestampMonotonic": "16117627", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-05-18 21:34:07 UTC", "ConditionTimestampMonotonic": "16117627", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup" 2020-05-18T21:36:43.593 INFO:teuthology.task.ansible.out:: "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "Documentation": "https://httpd.apache.org/docs/2.4/", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1164", "ExecMainStartTimestamp": "Mon 2020-05-18 21:34:08 UTC", "ExecMainStartTimestampMonotonic": "17399897", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "1844674407370 2020-05-18T21:36:43.595 INFO:teuthology.task.ansible.out:9551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-05-18 21:34:07 UTC", "InactiveExitTimestampMonotonic": "16118639", "InvocationID": "f55809a996bd4492b4f8743a0c513600", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127783", "LimitNPROCSoft": "127783", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127783", "LimitSIGPENDINGSoft": "127783", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "1164", "MemoryAccounting": "yes", "MemoryCurrent": "10395648", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPol 2020-05-18T21:36:43.597 INFO:teuthology.task.ansible.out:icy": "n/a", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-05-18 21:34:08 UTC", "StateChangeTimestampMonotonic": "17399936", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "38334", "TimeoutAbortUSec" 2020-05-18T21:36:43.598 INFO:teuthology.task.ansible.out:: "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-18T21:36:43.643 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:43 +0000 (0:00:00.203) 0:01:59.692 ************ 2020-05-18T21:36:43.688 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-05-18T21:36:43.690 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:43.726 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:43 +0000 (0:00:00.082) 0:01:59.775 ************ 2020-05-18T21:36:43.742 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* changed: [smithi080.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Mon 2020-05-18 21:34:16 UTC", "ActiveEnterTimestampMonotonic": "17841164", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service systemd-journald.socket network.target basic.target system.slice remote-fs.target sysinit.target nss-lookup.target -.mount", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-05-18 21:34:14 UTC", "AssertTimestampMonotonic": "16597805", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-05-18 21:34:14 UTC", "ConditionTimestampMonotonic": "16597805", 2020-05-18T21:36:43.743 INFO:teuthology.task.ansible.out:"ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "Documentation": "https://httpd.apache.org/docs/2.4/", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1166", "ExecMainStartTimestamp": "Mon 2020-05-18 21:34:16 UTC", "ExecMainStartTimestampMonotonic": "17841103", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", 2020-05-18T21:36:43.745 INFO:teuthology.task.ansible.out:"IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-05-18 21:34:14 UTC", "InactiveExitTimestampMonotonic": "16599964", "InvocationID": "82bf7ae660e147f59ff69032df1135f3", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127783", "LimitNPROCSoft": "127783", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127783", "LimitSIGPENDINGSoft": "127783", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "1166", "MemoryAccounting": "yes", "MemoryCurrent": "10100736", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "Moun 2020-05-18T21:36:43.747 INFO:teuthology.task.ansible.out:tFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-05-18 21:34:16 UTC", "StateChangeTimestampMonotonic": "17841164", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "Ta 2020-05-18T21:36:43.748 INFO:teuthology.task.ansible.out:sksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "38334", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-18T21:36:43.783 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:43 +0000 (0:00:00.057) 0:01:59.832 ************ 2020-05-18T21:36:43.792 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-05-18T21:36:43.794 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:43.831 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:43 +0000 (0:00:00.048) 0:01:59.880 ************ 2020-05-18T21:36:43.839 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-05-18T21:36:43.840 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:43.883 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:43 +0000 (0:00:00.051) 0:01:59.932 ************ 2020-05-18T21:36:43.891 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-05-18T21:36:43.893 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:43.931 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:43 +0000 (0:00:00.047) 0:01:59.979 ************ 2020-05-18T21:36:43.938 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-05-18T21:36:43.940 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:43.977 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:43 +0000 (0:00:00.046) 0:02:00.026 ************ 2020-05-18T21:36:43.986 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-05-18T21:36:43.987 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.023 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.046) 0:02:00.072 ************ 2020-05-18T21:36:44.033 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-05-18T21:36:44.035 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.079 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.053) 0:02:00.126 ************ 2020-05-18T21:36:44.087 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-05-18T21:36:44.089 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.126 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.048) 0:02:00.174 ************ 2020-05-18T21:36:44.135 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-05-18T21:36:44.137 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.173 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.047) 0:02:00.222 ************ 2020-05-18T21:36:44.182 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-05-18T21:36:44.184 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.221 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.047) 0:02:00.270 ************ 2020-05-18T21:36:44.228 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-05-18T21:36:44.230 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.267 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.045) 0:02:00.316 ************ 2020-05-18T21:36:44.275 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-05-18T21:36:44.277 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.314 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.047) 0:02:00.363 ************ 2020-05-18T21:36:44.323 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-05-18T21:36:44.325 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.361 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.046) 0:02:00.410 ************ 2020-05-18T21:36:44.370 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-05-18T21:36:44.372 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.414 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.052) 0:02:00.462 ************ 2020-05-18T21:36:44.422 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-05-18T21:36:44.425 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.461 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.046) 0:02:00.509 ************ 2020-05-18T21:36:44.470 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-18T21:36:44.471 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.508 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.047) 0:02:00.557 ************ 2020-05-18T21:36:44.516 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-05-18T21:36:44.518 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.554 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.046) 0:02:00.603 ************ 2020-05-18T21:36:44.563 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-05-18T21:36:44.565 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.607 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.052) 0:02:00.656 ************ 2020-05-18T21:36:44.615 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-18T21:36:44.617 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.653 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.046) 0:02:00.702 ************ 2020-05-18T21:36:44.663 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-05-18T21:36:44.665 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.701 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.047) 0:02:00.750 ************ 2020-05-18T21:36:44.709 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-05-18T21:36:44.710 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.747 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.045) 0:02:00.795 ************ 2020-05-18T21:36:44.755 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-05-18T21:36:44.757 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.793 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.046) 0:02:00.842 ************ 2020-05-18T21:36:44.802 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-05-18T21:36:44.804 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.840 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.046) 0:02:00.889 ************ 2020-05-18T21:36:44.848 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-05-18T21:36:44.850 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.886 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.046) 0:02:00.935 ************ 2020-05-18T21:36:44.894 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-05-18T21:36:44.896 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.931 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.045) 0:02:00.980 ************ 2020-05-18T21:36:44.940 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-18T21:36:44.942 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:44.979 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:44 +0000 (0:00:00.046) 0:02:01.027 ************ 2020-05-18T21:36:44.987 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.050 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.045) 0:02:01.073 ************ TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.072 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.048) 0:02:01.121 ************ 2020-05-18T21:36:45.080 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-18T21:36:45.082 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.120 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.046) 0:02:01.168 ************ 2020-05-18T21:36:45.130 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-18T21:36:45.132 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.169 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.050) 0:02:01.218 ************ 2020-05-18T21:36:45.177 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-05-18T21:36:45.179 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.226 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.056) 0:02:01.275 ************ 2020-05-18T21:36:45.236 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-05-18T21:36:45.238 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.275 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.048) 0:02:01.324 ************ 2020-05-18T21:36:45.283 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-18T21:36:45.285 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.322 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.046) 0:02:01.370 ************ 2020-05-18T21:36:45.330 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-05-18T21:36:45.332 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.369 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.047) 0:02:01.418 ************ 2020-05-18T21:36:45.377 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-05-18T21:36:45.379 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.416 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.046) 0:02:01.465 ************ 2020-05-18T21:36:45.425 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-05-18T21:36:45.427 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.464 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.048) 0:02:01.513 ************ 2020-05-18T21:36:45.472 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-05-18T21:36:45.474 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.512 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.048) 0:02:01.561 ************ 2020-05-18T21:36:45.520 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-18T21:36:45.522 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.559 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.046) 0:02:01.608 ************ 2020-05-18T21:36:45.566 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-05-18T21:36:45.568 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.608 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.049) 0:02:01.657 ************ 2020-05-18T21:36:45.617 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-05-18T21:36:45.618 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.656 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.047) 0:02:01.705 ************ 2020-05-18T21:36:45.670 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.702 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.046) 0:02:01.751 ************ 2020-05-18T21:36:45.712 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-18T21:36:45.714 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.751 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.048) 0:02:01.800 ************ 2020-05-18T21:36:45.760 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-05-18T21:36:45.762 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.799 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.047) 0:02:01.848 ************ 2020-05-18T21:36:45.808 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-18T21:36:45.810 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.847 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.048) 0:02:01.896 ************ 2020-05-18T21:36:45.856 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-18T21:36:45.858 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.898 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.050) 0:02:01.946 ************ 2020-05-18T21:36:45.908 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-05-18T21:36:45.909 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.946 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.048) 0:02:01.995 ************ 2020-05-18T21:36:45.955 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-18T21:36:45.957 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:45.993 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:45 +0000 (0:00:00.046) 0:02:02.041 ************ 2020-05-18T21:36:46.001 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-18T21:36:46.004 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:46.039 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:46 +0000 (0:00:00.046) 0:02:02.088 ************ 2020-05-18T21:36:46.047 INFO:teuthology.task.ansible.out: 2020-05-18T21:36:46.049 INFO:teuthology.task.ansible.out:TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:46.086 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:46 +0000 (0:00:00.046) 0:02:02.134 ************ 2020-05-18T21:36:46.094 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-05-18T21:36:46.096 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:46.132 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:46 +0000 (0:00:00.046) 0:02:02.181 ************ 2020-05-18T21:36:46.139 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-18T21:36:46.141 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:46.178 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:46 +0000 (0:00:00.045) 0:02:02.227 ************ 2020-05-18T21:36:46.209 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-05-18T21:36:46.211 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:46.248 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:46 +0000 (0:00:00.070) 0:02:02.297 ************ 2020-05-18T21:36:46.277 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-05-18T21:36:46.279 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-05-18T21:36:46.315 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:46 +0000 (0:00:00.066) 0:02:02.364 ************ 2020-05-18T21:36:46.386 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-05-18T21:36:46.422 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:46 +0000 (0:00:00.106) 0:02:02.471 ************ 2020-05-18T21:36:46.669 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-05-18T21:36:46.671 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-05-18T21:36:46.715 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:46 +0000 (0:00:00.293) 0:02:02.764 ************ 2020-05-18T21:36:46.746 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-05-18T21:36:46.747 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:46.788 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:46 +0000 (0:00:00.072) 0:02:02.837 ************ 2020-05-18T21:36:46.806 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-05-18T21:36:46.808 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-05-18T21:36:46.851 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:46 +0000 (0:00:00.062) 0:02:02.899 ************ 2020-05-18T21:36:46.894 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-05-18T21:36:46.895 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:46.934 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:46 +0000 (0:00:00.083) 0:02:02.983 ************ 2020-05-18T21:36:47.159 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-05-18T21:36:47.166 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-05-18T21:36:47.199 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:47 +0000 (0:00:00.265) 0:02:03.248 ************ 2020-05-18T21:36:47.322 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-05-18T21:36:47.374 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:47 +0000 (0:00:00.174) 0:02:03.423 ************ 2020-05-18T21:36:47.635 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-05-18T21:36:47.637 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-05-18T21:36:47.682 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:47 +0000 (0:00:00.308) 0:02:03.731 ************ 2020-05-18T21:36:47.776 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-05-18T21:36:47.815 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:47 +0000 (0:00:00.133) 0:02:03.864 ************ 2020-05-18T21:36:48.034 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-05-18T21:36:48.036 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-05-18T21:36:48.081 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.266) 0:02:04.130 ************ 2020-05-18T21:36:48.196 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-05-18T21:36:48.235 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.153) 0:02:04.284 ************ 2020-05-18T21:36:48.244 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-05-18T21:36:48.246 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-18T21:36:48.293 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.058) 0:02:04.342 ************ 2020-05-18T21:36:48.325 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-05-18T21:36:48.327 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:48.365 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.071) 0:02:04.414 ************ 2020-05-18T21:36:48.394 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-05-18T21:36:48.396 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-18T21:36:48.431 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.066) 0:02:04.480 ************ 2020-05-18T21:36:48.442 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-05-18T21:36:48.443 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:48.482 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.050) 0:02:04.531 ************ 2020-05-18T21:36:48.491 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-05-18T21:36:48.492 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:48.533 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.050) 0:02:04.581 ************ 2020-05-18T21:36:48.541 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-05-18T21:36:48.543 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:48.583 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.050) 0:02:04.632 ************ 2020-05-18T21:36:48.592 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-05-18T21:36:48.594 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:48.632 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.048) 0:02:04.681 ************ 2020-05-18T21:36:48.641 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-05-18T21:36:48.643 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:48.691 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.059) 0:02:04.740 ************ 2020-05-18T21:36:48.700 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-05-18T21:36:48.702 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:48.756 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.064) 0:02:04.805 ************ 2020-05-18T21:36:48.766 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-05-18T21:36:48.768 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:48.804 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.048) 0:02:04.853 ************ 2020-05-18T21:36:48.812 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-05-18T21:36:48.814 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:48.852 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.047) 0:02:04.901 ************ 2020-05-18T21:36:48.862 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-18T21:36:48.864 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-05-18T21:36:48.900 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.047) 0:02:04.949 ************ 2020-05-18T21:36:48.908 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-05-18T21:36:48.910 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:48.946 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:48 +0000 (0:00:00.045) 0:02:04.994 ************ 2020-05-18T21:36:48.964 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-05-18T21:36:48.966 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:48.986 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi080.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-05-18T21:36:48.988 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi196.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.023 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:49 +0000 (0:00:00.077) 0:02:05.072 ************ 2020-05-18T21:36:49.032 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.033 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.035 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi196.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.045 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.059 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.060 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.071 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.121 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:49 +0000 (0:00:00.097) 0:02:05.169 ************ 2020-05-18T21:36:49.130 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.132 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.133 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.133 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.145 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-05-18T21:36:49.147 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.192 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.194 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.194 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.240 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:49 +0000 (0:00:00.119) 0:02:05.289 ************ 2020-05-18T21:36:49.385 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-05-18T21:36:49.387 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004721", "end": "2020-05-18 21:36:49.346746", "rc": 0, "start": "2020-05-18 21:36:49.342025", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-05-18T21:36:49.452 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:49 +0000 (0:00:00.211) 0:02:05.500 ************ 2020-05-18T21:36:49.482 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004128", "end": "2020-05-18 21:36:49.448895", "rc": 0, "start": "2020-05-18 21:36:49.444767", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-05-18T21:36:49.531 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:49 +0000 (0:00:00.079) 0:02:05.580 ************ 2020-05-18T21:36:49.539 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-05-18T21:36:49.541 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.588 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:49 +0000 (0:00:00.057) 0:02:05.637 ************ 2020-05-18T21:36:49.600 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.643 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:49 +0000 (0:00:00.054) 0:02:05.691 ************ 2020-05-18T21:36:49.652 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-05-18T21:36:49.654 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.695 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:49 +0000 (0:00:00.051) 0:02:05.743 ************ 2020-05-18T21:36:49.706 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.746 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:49 +0000 (0:00:00.051) 0:02:05.795 ************ 2020-05-18T21:36:49.757 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-05-18T21:36:49.759 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.798 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:49 +0000 (0:00:00.052) 0:02:05.847 ************ 2020-05-18T21:36:49.809 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:49.849 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:49 +0000 (0:00:00.050) 0:02:05.898 ************ 2020-05-18T21:36:50.413 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* ok: [smithi196.front.sepia.ceph.com] => {"cache_update_time": 1589837434, "cache_updated": false, "changed": false} 2020-05-18T21:36:50.451 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:50 +0000 (0:00:00.599) 0:02:06.497 ************ 2020-05-18T21:36:50.470 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"cache_update_time": 1589837492, "cache_updated": false, "changed": false} 2020-05-18T21:36:50.508 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:50 +0000 (0:00:00.059) 0:02:06.556 ************ 2020-05-18T21:36:50.523 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-05-18T21:36:50.525 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:50.564 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:50 +0000 (0:00:00.056) 0:02:06.613 ************ 2020-05-18T21:36:50.574 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:50.617 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:50 +0000 (0:00:00.053) 0:02:06.666 ************ 2020-05-18T21:36:50.628 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-05-18T21:36:50.630 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item={'block_used': 13938059, 'uuid': 'e74b88d4-7bc9-445a-bab7-598d7476c420', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226157049, 'size_available': 926339272704, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 163927, 'block_size': 4096, 'inode_available': 60891049}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226157049, "block_size": 4096, "block_total": 240095108, "block_used": 13938059, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60891049, "inode_total": 61054976, "inode_used": 163927, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926339272704, "size_total": 983429562368, "uuid": "e74b88d4-7bc9-445a-bab7-598d7476c420"}} 2020-05-18T21:36:50.672 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:50 +0000 (0:00:00.055) 0:02:06.721 ************ 2020-05-18T21:36:50.683 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item={'block_used': 13938072, 'uuid': 'e74b88d4-7bc9-445a-bab7-598d7476c420', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226157036, 'size_available': 926339219456, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 163945, 'block_size': 4096, 'inode_available': 60891031}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226157036, "block_size": 4096, "block_total": 240095108, "block_used": 13938072, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60891031, "inode_total": 61054976, "inode_used": 163945, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926339219456, "size_total": 983429562368, "uuid": "e74b88d4-7bc9-445a-bab7-598d7476c420"}} 2020-05-18T21:36:50.729 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:50 +0000 (0:00:00.056) 0:02:06.777 ************ 2020-05-18T21:36:50.835 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-05-18T21:36:50.837 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.008442", "end": "2020-05-18 21:36:51.234707", "rc": 0, "start": "2020-05-18 21:36:51.226265", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:36:50.874 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:50 +0000 (0:00:00.145) 0:02:06.923 ************ 2020-05-18T21:36:50.895 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.009419", "end": "2020-05-18 21:36:50.479117", "rc": 0, "start": "2020-05-18 21:36:50.469698", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:36:50.940 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:50 +0000 (0:00:00.066) 0:02:06.989 ************ 2020-05-18T21:36:50.950 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-05-18T21:36:50.952 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'block_used': 13938059, 'uuid': 'e74b88d4-7bc9-445a-bab7-598d7476c420', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226157049, 'size_available': 926339272704, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 163927, 'block_size': 4096, 'inode_available': 60891049}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226157049, "block_size": 4096, "block_total": 240095108, "block_used": 13938059, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60891049, "inode_total": 61054976, "inode_used": 163927, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926339272704, "size_total": 983429562368, "uuid": "e74b88d4-7bc9-445a-bab7-598d7476c420"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:50.991 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:50 +0000 (0:00:00.051) 0:02:07.040 ************ 2020-05-18T21:36:51.000 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'block_used': 13938072, 'uuid': 'e74b88d4-7bc9-445a-bab7-598d7476c420', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226157036, 'size_available': 926339219456, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 163945, 'block_size': 4096, 'inode_available': 60891031}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226157036, "block_size": 4096, "block_total": 240095108, "block_used": 13938072, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60891031, "inode_total": 61054976, "inode_used": 163945, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926339219456, "size_total": 983429562368, "uuid": "e74b88d4-7bc9-445a-bab7-598d7476c420"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:51.040 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:51 +0000 (0:00:00.048) 0:02:07.089 ************ 2020-05-18T21:36:51.047 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-05-18T21:36:51.085 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:51 +0000 (0:00:00.045) 0:02:07.134 ************ 2020-05-18T21:36:51.134 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:51 +0000 (0:00:00.048) 0:02:07.182 ************ 2020-05-18T21:36:51.154 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:51.196 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:52.818 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sSypNEcYduvjMk2L5Bm0C8mBOTtJnnAtRo'], 'uuids': ['f1862588-48cc-4f2d-92f1-743f91ee9ae9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.537561", "end": "2020-05-18 21:36:53.220855", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sSypNEcYduvjMk2L5Bm0C8mBOTtJnnAtRo"], "labels": [], "masters": [], "uuids": ["f1862588-48cc-4f2d-92f1-743f91ee9ae9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:51.683294", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-18T21:36:52.830 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:52.841 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:52.842 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6p61ZZnfzlNMYy326f5dPCaTEGB3XwSJtX'], 'uuids': ['c231e266-0a6d-428a-bd8a-c8fc060141d9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.507014", "end": "2020-05-18 21:36:52.422275", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6p61ZZnfzlNMYy326f5dPCaTEGB3XwSJtX"], "labels": [], "masters": [], "uuids": ["c231e266-0a6d-428a-bd8a-c8fc060141d9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:50.915261", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-18T21:36:52.854 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:52.855 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:54.124 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sSrt054gP9Wfgu1MrhO8lYaiFTx7j3fTTG'], 'uuids': ['9b7f59db-71ad-4dd5-8e59-ea9d0d3cf435']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.156003", "end": "2020-05-18 21:36:54.526604", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sSrt054gP9Wfgu1MrhO8lYaiFTx7j3fTTG"], "labels": [], "masters": [], "uuids": ["9b7f59db-71ad-4dd5-8e59-ea9d0d3cf435"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:53.370601", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-18T21:36:54.135 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6pvdM7eBZXAfUy4vrl74DDX2DN8n5Z4bLR'], 'uuids': ['c069cd6d-3149-4f26-a0d6-9ce8dd0e6c0f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.141501", "end": "2020-05-18 21:36:53.712933", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6pvdM7eBZXAfUy4vrl74DDX2DN8n5Z4bLR"], "labels": [], "masters": [], "uuids": ["c069cd6d-3149-4f26-a0d6-9ce8dd0e6c0f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:52.571432", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-18T21:36:54.138 INFO:teuthology.task.ansible.out: skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XGPQ', 'wwn-0x5000c50092928bab'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'e74b88d4-7bc9-445a-bab7-598d7476c420', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XGPQ-part1', 'wwn-0x5000c50092928bab-part1'], 'uuids': ['e74b88d4-7bc9-445a-bab7-598d7476c420']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50092928bab', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XGPQ', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XGPQ", "wwn-0x5000c50092928bab"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XGPQ-part1", "wwn-0x5000c50092928bab-part1"], "labels": [], "masters": [], "uuids": ["e74b88d4-7bc9-445a-bab7-598d7476c420"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "e74b88d4-7bc9-445a-bab7-598d7476c420"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XGPQ", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092928bab"} 2020-05-18T21:36:54.139 INFO:teuthology.task.ansible.out:}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:54.141 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5R0PW', 'wwn-0x5000c50091e2e714'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'e74b88d4-7bc9-445a-bab7-598d7476c420', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5R0PW-part1', 'wwn-0x5000c50091e2e714-part1'], 'uuids': ['e74b88d4-7bc9-445a-bab7-598d7476c420']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091e2e714', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5R0PW', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5R0PW", "wwn-0x5000c50091e2e714"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5R0PW-part1", "wwn-0x5000c50091e2e714-part1"], "labels": [], "masters": [], "uuids": ["e74b88d4-7bc9-445a-bab7-598d7476c420"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "e74b88d4-7bc9-445a-bab7-598d7476c420"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5R0PW", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e2e714"}}, 2020-05-18T21:36:54.142 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-05-18T21:36:55.423 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-oCHony-vEMC-WbcP-sK9Q-d6zj-Uejq-IF4en9', 'nvme-INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN', 'nvme-nvme.8086-43564654363234363030305a34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_4', 'vg_nvme-lv_1', 'vg_nvme-lv_3', 'vg_nvme-lv_5', 'vg_nvme-lv_2'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.161015", "end": "2020-05-18 21:36:55.827209", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_4", "vg_nvme-lv_1", "vg_nvme-lv_3", "vg_nvme-lv_5", "vg_nvme-lv_2"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-oCHony-vEMC-WbcP-sK9Q-d6zj-Uejq-IF4en9", "nvme-INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN", "nvme-nvme.8086-43564654363234363030305a34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:54.666194", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nGPT data structures d 2020-05-18T21:36:55.425 INFO:teuthology.task.ansible.out:estroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-18T21:36:55.435 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:55.436 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:55.490 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-RN7SB2-tcmG-PK3H-WfPV-yYDC-AfBz-aoO7Z2', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204014T400BGN', 'nvme-nvme.8086-50484654363230343031345434303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_4', 'vg_nvme-lv_1', 'vg_nvme-lv_3', 'vg_nvme-lv_5', 'vg_nvme-lv_2'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.223593", "end": "2020-05-18 21:36:55.072607", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_4", "vg_nvme-lv_1", "vg_nvme-lv_3", "vg_nvme-lv_5", "vg_nvme-lv_2"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-RN7SB2-tcmG-PK3H-WfPV-yYDC-AfBz-aoO7Z2", "nvme-INTEL_SSDPEDMD400G4_PHFT6204014T400BGN", "nvme-nvme.8086-50484654363230343031345434303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:53.849014", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nGPT data structures d 2020-05-18T21:36:55.492 INFO:teuthology.task.ansible.out:estroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-18T21:36:55.512 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:55.513 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:56.738 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sS0ud0t7Dab0SKOgbP0zsAEl1fgFzzCmum'], 'uuids': ['29de88cc-a533-4140-b151-ba2c33507200']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.164754", "end": "2020-05-18 21:36:57.135380", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sS0ud0t7Dab0SKOgbP0zsAEl1fgFzzCmum"], "labels": [], "masters": [], "uuids": ["29de88cc-a533-4140-b151-ba2c33507200"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:55.970626", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-18T21:36:56.747 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:56.758 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:56.791 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6pKGx0HDDJuJyczqYxvfztO0EyTCCt0o9I'], 'uuids': ['c20add64-925c-4741-a8d1-cd8dea35b428']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.142164", "end": "2020-05-18 21:36:56.372845", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6pKGx0HDDJuJyczqYxvfztO0EyTCCt0o9I"], "labels": [], "masters": [], "uuids": ["c20add64-925c-4741-a8d1-cd8dea35b428"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:55.230681", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-18T21:36:56.813 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:56.814 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:58.017 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sSW7ZebHLcCJCVeRpKCyDTZexeSOwPSIlb'], 'uuids': ['caf57c4a-8748-44d6-88e5-ecb21306386b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.131041", "end": "2020-05-18 21:36:58.419152", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sSW7ZebHLcCJCVeRpKCyDTZexeSOwPSIlb"], "labels": [], "masters": [], "uuids": ["caf57c4a-8748-44d6-88e5-ecb21306386b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:57.288111", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-18T21:36:58.029 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:58.093 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6piedjwGN0X0ZQAZbSSpx9gfBxt3VcXkaz'], 'uuids': ['a4e9aca3-9b2c-4618-8b7a-dde1670d86b2']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.151131", "end": "2020-05-18 21:36:57.672769", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6piedjwGN0X0ZQAZbSSpx9gfBxt3VcXkaz"], "labels": [], "masters": [], "uuids": ["a4e9aca3-9b2c-4618-8b7a-dde1670d86b2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:56.521638", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-18T21:36:58.104 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:59.289 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sS6eUKNjL1i44vrilGORx0CJMgf6DPOEQr'], 'uuids': ['4ca4f0a0-39b2-4667-9d2a-3b6b33eeef54']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.127897", "end": "2020-05-18 21:36:59.685790", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sS6eUKNjL1i44vrilGORx0CJMgf6DPOEQr"], "labels": [], "masters": [], "uuids": ["4ca4f0a0-39b2-4667-9d2a-3b6b33eeef54"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:58.557893", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-18T21:36:59.350 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:59 +0000 (0:00:08.216) 0:02:15.399 ************ 2020-05-18T21:36:59.382 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6pP96Ut5OldrH5R9KitqZWiQhO4QrgprAj'], 'uuids': ['e127fc74-1aff-4c32-a111-94f103bf32a1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.133231", "end": "2020-05-18 21:36:58.948642", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6pP96Ut5OldrH5R9KitqZWiQhO4QrgprAj"], "labels": [], "masters": [], "uuids": ["e127fc74-1aff-4c32-a111-94f103bf32a1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:57.815411", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-18T21:36:59.431 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:36:59 +0000 (0:00:00.080) 0:02:15.479 ************ 2020-05-18T21:36:59.439 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-05-18T21:36:59.440 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:59.484 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:59.732 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sSypNEcYduvjMk2L5Bm0C8mBOTtJnnAtRo'], 'uuids': ['f1862588-48cc-4f2d-92f1-743f91ee9ae9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.179918", "end": "2020-05-18 21:37:00.124532", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sSypNEcYduvjMk2L5Bm0C8mBOTtJnnAtRo"], "labels": [], "masters": [], "uuids": ["f1862588-48cc-4f2d-92f1-743f91ee9ae9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:59.944614", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.15658 s, 737 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.15658 s, 737 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-18T21:36:59.743 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:59.755 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:59.797 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6p61ZZnfzlNMYy326f5dPCaTEGB3XwSJtX'], 'uuids': ['c231e266-0a6d-428a-bd8a-c8fc060141d9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.178335", "end": "2020-05-18 21:36:59.374461", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6p61ZZnfzlNMYy326f5dPCaTEGB3XwSJtX"], "labels": [], "masters": [], "uuids": ["c231e266-0a6d-428a-bd8a-c8fc060141d9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:59.196126", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.166358 s, 693 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.166358 s, 693 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-18T21:36:59.809 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:36:59.810 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:00.043 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sSrt054gP9Wfgu1MrhO8lYaiFTx7j3fTTG'], 'uuids': ['9b7f59db-71ad-4dd5-8e59-ea9d0d3cf435']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.166117", "end": "2020-05-18 21:37:00.448628", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sSrt054gP9Wfgu1MrhO8lYaiFTx7j3fTTG"], "labels": [], "masters": [], "uuids": ["9b7f59db-71ad-4dd5-8e59-ea9d0d3cf435"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:37:00.282511", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.161347 s, 715 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.161347 s, 715 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:00.054 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XGPQ', 'wwn-0x5000c50092928bab'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'e74b88d4-7bc9-445a-bab7-598d7476c420', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XGPQ-part1', 'wwn-0x5000c50092928bab-part1'], 'uuids': ['e74b88d4-7bc9-445a-bab7-598d7476c420']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50092928bab', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XGPQ', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XGPQ", "wwn-0x5000c50092928bab"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XGPQ-part1", "wwn-0x5000c50092928bab-part1"], "labels": [], "masters": [], "uuids": ["e74b88d4-7bc9-445a-bab7-598d7476c420"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "e74b88d4-7bc9-445a-bab7-598d7476c420"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XGPQ", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092928bab"}}, 2020-05-18T21:37:00.056 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-05-18T21:37:00.107 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6pvdM7eBZXAfUy4vrl74DDX2DN8n5Z4bLR'], 'uuids': ['c069cd6d-3149-4f26-a0d6-9ce8dd0e6c0f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.164768", "end": "2020-05-18 21:36:59.686541", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6pvdM7eBZXAfUy4vrl74DDX2DN8n5Z4bLR"], "labels": [], "masters": [], "uuids": ["c069cd6d-3149-4f26-a0d6-9ce8dd0e6c0f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:59.521773", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.159953 s, 721 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.159953 s, 721 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:00.123 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5R0PW', 'wwn-0x5000c50091e2e714'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'e74b88d4-7bc9-445a-bab7-598d7476c420', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5R0PW-part1', 'wwn-0x5000c50091e2e714-part1'], 'uuids': ['e74b88d4-7bc9-445a-bab7-598d7476c420']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091e2e714', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5R0PW', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5R0PW", "wwn-0x5000c50091e2e714"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5R0PW-part1", "wwn-0x5000c50091e2e714-part1"], "labels": [], "masters": [], "uuids": ["e74b88d4-7bc9-445a-bab7-598d7476c420"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "e74b88d4-7bc9-445a-bab7-598d7476c420"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5R0PW", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e2e714"}}, 2020-05-18T21:37:00.124 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-05-18T21:37:00.240 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-oCHony-vEMC-WbcP-sK9Q-d6zj-Uejq-IF4en9', 'nvme-INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN', 'nvme-nvme.8086-43564654363234363030305a34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_4', 'vg_nvme-lv_1', 'vg_nvme-lv_3', 'vg_nvme-lv_5', 'vg_nvme-lv_2'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.056528", "end": "2020-05-18 21:37:00.644226", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_4", "vg_nvme-lv_1", "vg_nvme-lv_3", "vg_nvme-lv_5", "vg_nvme-lv_2"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-oCHony-vEMC-WbcP-sK9Q-d6zj-Uejq-IF4en9", "nvme-INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN", "nvme-nvme.8086-43564654363234363030305a34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:37:00.587698", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0534215 s, 2.2 GB/s", "std 2020-05-18T21:37:00.242 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0534215 s, 2.2 GB/s"], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:00.262 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:00.263 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:00.306 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-RN7SB2-tcmG-PK3H-WfPV-yYDC-AfBz-aoO7Z2', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204014T400BGN', 'nvme-nvme.8086-50484654363230343031345434303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_4', 'vg_nvme-lv_1', 'vg_nvme-lv_3', 'vg_nvme-lv_5', 'vg_nvme-lv_2'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.056040", "end": "2020-05-18 21:36:59.890257", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_4", "vg_nvme-lv_1", "vg_nvme-lv_3", "vg_nvme-lv_5", "vg_nvme-lv_2"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-RN7SB2-tcmG-PK3H-WfPV-yYDC-AfBz-aoO7Z2", "nvme-INTEL_SSDPEDMD400G4_PHFT6204014T400BGN", "nvme-nvme.8086-50484654363230343031345434303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:36:59.834217", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.051185 s, 2.3 GB/s", "stde 2020-05-18T21:37:00.308 INFO:teuthology.task.ansible.out:rr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.051185 s, 2.3 GB/s"], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:00.319 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:00.333 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:00.549 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sS0ud0t7Dab0SKOgbP0zsAEl1fgFzzCmum'], 'uuids': ['29de88cc-a533-4140-b151-ba2c33507200']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.157551", "end": "2020-05-18 21:37:00.947213", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sS0ud0t7Dab0SKOgbP0zsAEl1fgFzzCmum"], "labels": [], "masters": [], "uuids": ["29de88cc-a533-4140-b151-ba2c33507200"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:37:00.789662", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152794 s, 755 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152794 s, 755 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:00.561 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:00.574 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:00.644 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6pKGx0HDDJuJyczqYxvfztO0EyTCCt0o9I'], 'uuids': ['c20add64-925c-4741-a8d1-cd8dea35b428']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.169826", "end": "2020-05-18 21:37:00.219293", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6pKGx0HDDJuJyczqYxvfztO0EyTCCt0o9I"], "labels": [], "masters": [], "uuids": ["c20add64-925c-4741-a8d1-cd8dea35b428"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:37:00.049467", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.165917 s, 695 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.165917 s, 695 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:00.651 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:00.664 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:00.857 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sSW7ZebHLcCJCVeRpKCyDTZexeSOwPSIlb'], 'uuids': ['caf57c4a-8748-44d6-88e5-ecb21306386b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.162114", "end": "2020-05-18 21:37:01.263292", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sSW7ZebHLcCJCVeRpKCyDTZexeSOwPSIlb"], "labels": [], "masters": [], "uuids": ["caf57c4a-8748-44d6-88e5-ecb21306386b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:37:01.101178", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.157346 s, 733 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.157346 s, 733 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:00.879 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:00.963 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6piedjwGN0X0ZQAZbSSpx9gfBxt3VcXkaz'], 'uuids': ['a4e9aca3-9b2c-4618-8b7a-dde1670d86b2']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.163146", "end": "2020-05-18 21:37:00.536847", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6piedjwGN0X0ZQAZbSSpx9gfBxt3VcXkaz"], "labels": [], "masters": [], "uuids": ["a4e9aca3-9b2c-4618-8b7a-dde1670d86b2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:37:00.373701", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.158409 s, 728 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.158409 s, 728 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:00.974 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:01.182 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sS6eUKNjL1i44vrilGORx0CJMgf6DPOEQr'], 'uuids': ['4ca4f0a0-39b2-4667-9d2a-3b6b33eeef54']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.175056", "end": "2020-05-18 21:37:01.578952", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-5TvGDtna0LgFYEMc3wi2oybyFl9eL0sS6eUKNjL1i44vrilGORx0CJMgf6DPOEQr"], "labels": [], "masters": [], "uuids": ["4ca4f0a0-39b2-4667-9d2a-3b6b33eeef54"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:37:01.403896", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.170158 s, 678 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.170158 s, 678 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:01.230 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:01 +0000 (0:00:01.799) 0:02:17.279 ************ 2020-05-18T21:37:01.274 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6pP96Ut5OldrH5R9KitqZWiQhO4QrgprAj'], 'uuids': ['e127fc74-1aff-4c32-a111-94f103bf32a1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.174652", "end": "2020-05-18 21:37:00.853651", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-6ys51qQYdH0c7TMrNqScVbVl17jh5i6pP96Ut5OldrH5R9KitqZWiQhO4QrgprAj"], "labels": [], "masters": [], "uuids": ["e127fc74-1aff-4c32-a111-94f103bf32a1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-18 21:37:00.678999", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.169739 s, 680 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.169739 s, 680 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:01.321 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:01 +0000 (0:00:00.090) 0:02:17.370 ************ 2020-05-18T21:37:01.510 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-05-18T21:37:01.512 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.124408", "end": "2020-05-18 21:37:01.907257", "rc": 0, "start": "2020-05-18 21:37:01.782849", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:01.548 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:01 +0000 (0:00:00.227) 0:02:17.597 ************ 2020-05-18T21:37:01.630 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.144379", "end": "2020-05-18 21:37:01.204614", "rc": 0, "start": "2020-05-18 21:37:01.060235", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:01.669 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:01 +0000 (0:00:00.120) 0:02:17.717 ************ 2020-05-18T21:37:01.796 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-05-18T21:37:01.798 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.092930", "end": "2020-05-18 21:37:02.207991", "rc": 0, "start": "2020-05-18 21:37:02.115061", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:01.840 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:01 +0000 (0:00:00.171) 0:02:17.889 ************ 2020-05-18T21:37:01.869 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-05-18T21:37:01.910 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:01 +0000 (0:00:00.070) 0:02:17.959 ************ 2020-05-18T21:37:01.927 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-05-18T21:37:01.929 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.074280", "end": "2020-05-18 21:37:01.477192", "rc": 0, "start": "2020-05-18 21:37:01.402912", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:01.976 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:01 +0000 (0:00:00.065) 0:02:18.024 ************ 2020-05-18T21:37:01.984 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-05-18T21:37:01.986 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.026 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.049) 0:02:18.074 ************ 2020-05-18T21:37:02.033 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-05-18T21:37:02.072 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.046) 0:02:18.121 ************ 2020-05-18T21:37:02.079 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-05-18T21:37:02.082 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.117 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.044) 0:02:18.166 ************ 2020-05-18T21:37:02.124 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-05-18T21:37:02.126 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.174 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.056) 0:02:18.222 ************ 2020-05-18T21:37:02.184 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-05-18T21:37:02.186 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.227 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.053) 0:02:18.276 ************ 2020-05-18T21:37:02.237 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-05-18T21:37:02.239 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.275 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.047) 0:02:18.324 ************ 2020-05-18T21:37:02.286 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-05-18T21:37:02.288 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.325 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.049) 0:02:18.373 ************ 2020-05-18T21:37:02.334 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-05-18T21:37:02.336 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.383 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.057) 0:02:18.431 ************ 2020-05-18T21:37:02.434 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-05-18T21:37:02.436 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.470 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.086) 0:02:18.518 ************ 2020-05-18T21:37:02.479 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-05-18T21:37:02.481 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-18T21:37:02.518 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.048) 0:02:18.567 ************ 2020-05-18T21:37:02.568 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-05-18T21:37:02.570 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'block_used': 13938059, 'uuid': 'e74b88d4-7bc9-445a-bab7-598d7476c420', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226157049, 'size_available': 926339272704, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 163927, 'block_size': 4096, 'inode_available': 60891049}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226157049, "block_size": 4096, "block_total": 240095108, "block_used": 13938059, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60891049, "inode_total": 61054976, "inode_used": 163927, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926339272704, "size_total": 983429562368, "uuid": "e74b88d4-7bc9-445a-bab7-598d7476c420"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.614 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.096) 0:02:18.663 ************ 2020-05-18T21:37:02.632 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-05-18T21:37:02.634 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-18T21:37:02.679 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.064) 0:02:18.728 ************ 2020-05-18T21:37:02.688 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-05-18T21:37:02.690 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.726 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.047) 0:02:18.775 ************ 2020-05-18T21:37:02.733 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-05-18T21:37:02.734 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'block_used': 13938072, 'uuid': 'e74b88d4-7bc9-445a-bab7-598d7476c420', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226157036, 'size_available': 926339219456, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 163945, 'block_size': 4096, 'inode_available': 60891031}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226157036, "block_size": 4096, "block_total": 240095108, "block_used": 13938072, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60891031, "inode_total": 61054976, "inode_used": 163945, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926339219456, "size_total": 983429562368, "uuid": "e74b88d4-7bc9-445a-bab7-598d7476c420"}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.775 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.049) 0:02:18.824 ************ 2020-05-18T21:37:02.783 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-18T21:37:02.785 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.821 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.045) 0:02:18.870 ************ 2020-05-18T21:37:02.829 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-05-18T21:37:02.831 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.869 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.047) 0:02:18.918 ************ 2020-05-18T21:37:02.879 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-18T21:37:02.881 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.921 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.051) 0:02:18.970 ************ 2020-05-18T21:37:02.931 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:02.970 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:02 +0000 (0:00:00.049) 0:02:19.019 ************ 2020-05-18T21:37:02.978 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-05-18T21:37:02.980 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:03.022 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:03 +0000 (0:00:00.052) 0:02:19.071 ************ 2020-05-18T21:37:03.033 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:03.078 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:03 +0000 (0:00:00.055) 0:02:19.127 ************ 2020-05-18T21:37:03.089 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-05-18T21:37:03.091 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:03.128 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:03 +0000 (0:00:00.049) 0:02:19.177 ************ 2020-05-18T21:37:03.135 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-05-18T21:37:03.137 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:03.173 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:03 +0000 (0:00:00.045) 0:02:19.222 ************ 2020-05-18T21:37:03.181 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-05-18T21:37:03.183 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:03.220 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:03 +0000 (0:00:00.047) 0:02:19.269 ************ 2020-05-18T21:37:03.232 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-05-18T21:37:03.234 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:03.273 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:03 +0000 (0:00:00.051) 0:02:19.321 ************ 2020-05-18T21:37:03.314 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-05-18T21:37:03.317 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:03.354 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:03 +0000 (0:00:00.081) 0:02:19.403 ************ 2020-05-18T21:37:04.146 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-05-18T21:37:04.148 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-05-18T21:37:04.190 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:04 +0000 (0:00:00.835) 0:02:20.239 ************ 2020-05-18T21:37:04.200 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-05-18T21:37:04.238 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:04 +0000 (0:00:00.048) 0:02:20.287 ************ 2020-05-18T21:37:04.938 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-05-18T21:37:04.940 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-18T21:37:04.991 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-18T21:37:04.993 INFO:teuthology.task.ansible.out: 2020-05-18T21:37:05.323 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-18T21:37:05.407 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-18T21:37:05.728 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-18T21:37:05.811 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-18T21:37:06.101 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-18T21:37:06.195 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-18T21:37:06.496 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-18T21:37:06.540 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:06 +0000 (0:00:02.302) 0:02:22.589 ************ 2020-05-18T21:37:06.603 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-18T21:37:06.650 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:06 +0000 (0:00:00.109) 0:02:22.699 ************ 2020-05-18T21:37:06.703 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-05-18T21:37:06.704 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-05-18T21:37:06.748 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:06 +0000 (0:00:00.097) 0:02:22.797 ************ 2020-05-18T21:37:06.795 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-05-18T21:37:06.836 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:06 +0000 (0:00:00.088) 0:02:22.885 ************ 2020-05-18T21:37:06.907 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-05-18T21:37:06.908 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-18T21:37:07.001 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-18T21:37:07.044 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-18T21:37:07.138 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-18T21:37:07.159 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-18T21:37:07.253 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-18T21:37:07.278 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-18T21:37:07.287 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:07.327 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:07 +0000 (0:00:00.490) 0:02:23.376 ************ 2020-05-18T21:37:07.368 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-18T21:37:07.379 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:07.420 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:07 +0000 (0:00:00.092) 0:02:23.469 ************ 2020-05-18T21:37:07.470 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-05-18T21:37:07.471 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-05-18T21:37:07.511 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:07 +0000 (0:00:00.090) 0:02:23.560 ************ 2020-05-18T21:37:07.551 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-05-18T21:37:07.553 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-05-18T21:37:07.591 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:07 +0000 (0:00:00.080) 0:02:23.640 ************ 2020-05-18T21:37:07.612 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-05-18T21:37:07.614 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-05-18T21:37:07.650 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:07 +0000 (0:00:00.059) 0:02:23.699 ************ 2020-05-18T21:37:07.692 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-05-18T21:37:07.694 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-05-18T21:37:07.729 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:07 +0000 (0:00:00.079) 0:02:23.778 ************ 2020-05-18T21:37:08.136 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-05-18T21:37:08.138 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"changed": true} 2020-05-18T21:37:08.186 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:08 +0000 (0:00:00.457) 0:02:24.235 ************ 2020-05-18T21:37:08.195 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"changed": true} 2020-05-18T21:37:08.234 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:08 +0000 (0:00:00.047) 0:02:24.282 ************ 2020-05-18T21:37:08.683 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-05-18T21:37:08.684 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-05-18T21:37:08.723 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:08 +0000 (0:00:00.489) 0:02:24.772 ************ 2020-05-18T21:37:08.734 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-05-18T21:37:08.772 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:08 +0000 (0:00:00.048) 0:02:24.821 ************ 2020-05-18T21:37:08.873 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "gid": 1214, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-05-18T21:37:08.910 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:08 +0000 (0:00:00.138) 0:02:24.959 ************ 2020-05-18T21:37:08.933 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "gid": 1214, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-05-18T21:37:08.969 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:08 +0000 (0:00:00.058) 0:02:25.018 ************ 2020-05-18T21:37:09.080 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-05-18T21:37:09.082 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-05-18T21:37:09.136 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:09 +0000 (0:00:00.166) 0:02:25.185 ************ 2020-05-18T21:37:09.145 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-05-18T21:37:09.218 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:09 +0000 (0:00:00.081) 0:02:25.267 ************ 2020-05-18T21:37:09.503 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-05-18T21:37:09.505 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1214, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-05-18T21:37:09.540 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:09 +0000 (0:00:00.321) 0:02:25.589 ************ 2020-05-18T21:37:09.590 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1214, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-05-18T21:37:09.645 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:09 +0000 (0:00:00.105) 0:02:25.694 ************ 2020-05-18T21:37:09.891 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-05-18T21:37:09.893 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-05-18T21:37:09.935 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:09 +0000 (0:00:00.289) 0:02:25.984 ************ 2020-05-18T21:37:10.005 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-05-18T21:37:10.044 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:10 +0000 (0:00:00.108) 0:02:26.092 ************ 2020-05-18T21:37:10.568 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-05-18T21:37:10.570 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"cache_update_time": 1589837434, "cache_updated": false, "changed": false} 2020-05-18T21:37:10.607 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:10 +0000 (0:00:00.564) 0:02:26.656 ************ 2020-05-18T21:37:10.659 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"cache_update_time": 1589837492, "cache_updated": false, "changed": false} 2020-05-18T21:37:10.699 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:10 +0000 (0:00:00.091) 0:02:26.747 ************ 2020-05-18T21:37:10.893 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-05-18T21:37:10.895 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.123976", "end": "2020-05-18 21:37:11.300624", "rc": 0, "start": "2020-05-18 21:37:11.176648", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.30.0/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.30.0/Amazon/S3.pm"]} 2020-05-18T21:37:10.932 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:10 +0000 (0:00:00.233) 0:02:26.981 ************ 2020-05-18T21:37:10.974 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-05-18T21:37:10.976 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:11.011 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:11 +0000 (0:00:00.079) 0:02:27.060 ************ 2020-05-18T21:37:11.019 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-05-18T21:37:11.021 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.119161", "end": "2020-05-18 21:37:10.585870", "rc": 0, "start": "2020-05-18 21:37:10.466709", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.30.0/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.30.0/Amazon/S3.pm"]} 2020-05-18T21:37:11.056 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:11 +0000 (0:00:00.044) 0:02:27.105 ************ 2020-05-18T21:37:11.064 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-05-18T21:37:11.065 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:11.101 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:11 +0000 (0:00:00.044) 0:02:27.150 ************ 2020-05-18T21:37:11.109 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-05-18T21:37:11.111 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:11.147 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:11 +0000 (0:00:00.045) 0:02:27.196 ************ 2020-05-18T21:37:11.178 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-05-18T21:37:11.180 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:11.214 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:11 +0000 (0:00:00.066) 0:02:27.263 ************ 2020-05-18T21:37:11.713 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-05-18T21:37:11.715 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"cache_update_time": 1589837434, "cache_updated": false, "changed": false} 2020-05-18T21:37:11.753 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:11 +0000 (0:00:00.538) 0:02:27.802 ************ 2020-05-18T21:37:11.802 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-05-18T21:37:11.804 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:11.844 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:11 +0000 (0:00:00.091) 0:02:27.893 ************ 2020-05-18T21:37:11.852 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-05-18T21:37:11.853 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"cache_update_time": 1589837492, "cache_updated": false, "changed": false} 2020-05-18T21:37:11.890 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:11 +0000 (0:00:00.045) 0:02:27.939 ************ 2020-05-18T21:37:11.921 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:11.953 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:11 +0000 (0:00:00.063) 0:02:28.002 ************ 2020-05-18T21:37:12.195 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-05-18T21:37:12.197 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-05-18T21:37:12.233 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:12 +0000 (0:00:00.280) 0:02:28.282 ************ 2020-05-18T21:37:12.262 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-05-18T21:37:12.264 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:12.299 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:12 +0000 (0:00:00.065) 0:02:28.348 ************ 2020-05-18T21:37:12.308 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-05-18T21:37:12.310 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-05-18T21:37:12.347 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:12 +0000 (0:00:00.047) 0:02:28.396 ************ 2020-05-18T21:37:12.385 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:12.421 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:12 +0000 (0:00:00.074) 0:02:28.470 ************ 2020-05-18T21:37:12.512 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-05-18T21:37:12.514 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-05-18 21:34:06 UTC", "ActiveEnterTimestampMonotonic": "14952778", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target -.mount basic.target systemd-tmpfiles-setup.service systemd-journald.socket sysinit.target system.slice", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-05-18 21:34:05 UTC", "AssertTimestampMonotonic": "14739021", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-05-18 21:34:05 UTC", "ConditionTimestampMonotonic": "14739020", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": 2020-05-18T21:37:12.515 INFO:teuthology.task.ansible.out: "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "944", "ExecMainStartTimestamp": "Mon 2020-05-18 21:34:06 UTC", "ExecMainStartTimestampMonotonic": "14952764", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-05-18 21:34:05 UTC", "InactiveExitTimestampMonotonic": "14740483", "InvocationID": "9ef3de1c5791435abd88676844d77968", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "Lim 2020-05-18T21:37:12.517 INFO:teuthology.task.ansible.out:itCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127783", "LimitNPROCSoft": "127783", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127783", "LimitSIGPENDINGSoft": "127783", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "944", "MemoryAccounting": "yes", "MemoryCurrent": "3584000", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "Requi 2020-05-18T21:37:12.519 INFO:teuthology.task.ansible.out:resMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-05-18 21:34:06 UTC", "StateChangeTimestampMonotonic": "14952778", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "38334", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-18T21:37:12.552 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:12 +0000 (0:00:00.130) 0:02:28.601 ************ 2020-05-18T21:37:12.595 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-18T21:37:12.597 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-05-18T21:37:12.632 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:12 +0000 (0:00:00.079) 0:02:28.681 ************ 2020-05-18T21:37:12.641 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-05-18T21:37:12.643 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-05-18 21:34:13 UTC", "ActiveEnterTimestampMonotonic": "15180288", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-journald.socket sysinit.target network.target basic.target systemd-tmpfiles-setup.service -.mount system.slice", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-05-18 21:34:13 UTC", "AssertTimestampMonotonic": "15063701", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-05-18 21:34:13 UTC", "ConditionTimestampMonotonic": "15063699", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": 2020-05-18T21:37:12.644 INFO:teuthology.task.ansible.out: "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "948", "ExecMainStartTimestamp": "Mon 2020-05-18 21:34:13 UTC", "ExecMainStartTimestampMonotonic": "15180275", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-05-18 21:34:13 UTC", "InactiveExitTimestampMonotonic": "15064696", "InvocationID": "a5cbcafdedb84c938cc9222fae2b5c67", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "Lim 2020-05-18T21:37:12.646 INFO:teuthology.task.ansible.out:itCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127783", "LimitNPROCSoft": "127783", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127783", "LimitSIGPENDINGSoft": "127783", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "948", "MemoryAccounting": "yes", "MemoryCurrent": "3518464", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "Requi 2020-05-18T21:37:12.647 INFO:teuthology.task.ansible.out:resMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-05-18 21:34:13 UTC", "StateChangeTimestampMonotonic": "15180288", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "38334", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-18T21:37:12.679 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:12 +0000 (0:00:00.047) 0:02:28.728 ************ 2020-05-18T21:37:12.687 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-18T21:37:12.689 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-05-18T21:37:12.725 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:12 +0000 (0:00:00.045) 0:02:28.774 ************ 2020-05-18T21:37:12.732 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-18T21:37:12.734 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-05-18T21:37:12.770 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:12 +0000 (0:00:00.045) 0:02:28.819 ************ 2020-05-18T21:37:12.777 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-05-18T21:37:12.779 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:12.819 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:12 +0000 (0:00:00.049) 0:02:28.868 ************ 2020-05-18T21:37:12.826 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-18T21:37:12.828 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-05-18T21:37:12.863 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:12 +0000 (0:00:00.044) 0:02:28.912 ************ 2020-05-18T21:37:12.891 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-05-18T21:37:12.893 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:12.927 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:12 +0000 (0:00:00.063) 0:02:28.975 ************ 2020-05-18T21:37:13.864 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-05-18T21:37:13.866 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"cache_update_time": 1589837434, "cache_updated": false, "changed": false} 2020-05-18T21:37:13.902 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:13 +0000 (0:00:00.975) 0:02:29.951 ************ 2020-05-18T21:37:13.930 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-05-18T21:37:13.932 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:13.966 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:13 +0000 (0:00:00.064) 0:02:30.015 ************ 2020-05-18T21:37:13.994 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-05-18T21:37:13.996 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"cache_update_time": 1589837492, "cache_updated": false, "changed": false} 2020-05-18T21:37:14.036 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:14 +0000 (0:00:00.069) 0:02:30.085 ************ 2020-05-18T21:37:14.065 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-05-18T21:37:14.067 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:14.101 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:14 +0000 (0:00:00.064) 0:02:30.150 ************ 2020-05-18T21:37:14.150 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-05-18T21:37:14.152 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "gid": 1214, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-05-18T21:37:14.188 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:14 +0000 (0:00:00.087) 0:02:30.237 ************ 2020-05-18T21:37:14.290 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "gid": 1214, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-05-18T21:37:14.328 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:14 +0000 (0:00:00.139) 0:02:30.377 ************ 2020-05-18T21:37:14.643 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-05-18T21:37:14.645 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1214, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-05-18T21:37:14.688 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:14 +0000 (0:00:00.360) 0:02:30.737 ************ 2020-05-18T21:37:14.815 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1214, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-05-18T21:37:14.865 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:14 +0000 (0:00:00.177) 0:02:30.914 ************ 2020-05-18T21:37:17.242 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)\nRequirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.7.1)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)", "Requirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.7.1)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)"], "version": null, "virtualenv": null} 2020-05-18T21:37:17.282 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:17 +0000 (0:00:02.417) 0:02:33.331 ************ 2020-05-18T21:37:17.306 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)\nRequirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.7.1)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)", "Requirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.7.1)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)"], "version": null, "virtualenv": null} 2020-05-18T21:37:17.352 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:17 +0000 (0:00:00.069) 0:02:33.400 ************ 2020-05-18T21:37:17.790 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-05-18T21:37:17.792 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false} 2020-05-18T21:37:17.829 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:17 +0000 (0:00:00.477) 0:02:33.878 ************ 2020-05-18T21:37:17.849 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false} 2020-05-18T21:37:17.888 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:17 +0000 (0:00:00.059) 0:02:33.937 ************ 2020-05-18T21:37:17.900 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-05-18T21:37:17.902 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-05-18T21:37:17.938 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:17 +0000 (0:00:00.049) 0:02:33.986 ************ 2020-05-18T21:37:17.946 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-05-18T21:37:17.984 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:17 +0000 (0:00:00.046) 0:02:34.033 ************ 2020-05-18T21:37:18.147 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-05-18T21:37:18.149 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1589837418.792001, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1589837418.788001, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 87101, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1589837418.788001, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "867956063", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-05-18T21:37:18.190 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:18 +0000 (0:00:00.206) 0:02:34.239 ************ 2020-05-18T21:37:18.200 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1589837411.8918924, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1589837411.8878925, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 87101, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1589837411.8878925, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "867956063", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-05-18T21:37:18.239 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:18 +0000 (0:00:00.048) 0:02:34.288 ************ 2020-05-18T21:37:18.646 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** ok: [smithi196.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-05-18T21:37:18.681 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:18 +0000 (0:00:00.441) 0:02:34.729 ************ 2020-05-18T21:37:18.690 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-05-18T21:37:18.734 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:18 +0000 (0:00:00.053) 0:02:34.783 ************ 2020-05-18T21:37:18.746 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-05-18T21:37:18.748 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-05-18T21:37:18.786 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:18 +0000 (0:00:00.051) 0:02:34.835 ************ 2020-05-18T21:37:18.797 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-05-18T21:37:18.836 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:18 +0000 (0:00:00.049) 0:02:34.885 ************ 2020-05-18T21:37:18.846 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-05-18T21:37:18.848 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:18.885 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:18 +0000 (0:00:00.048) 0:02:34.934 ************ 2020-05-18T21:37:18.895 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:18.943 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:18 +0000 (0:00:00.058) 0:02:34.992 ************ 2020-05-18T21:37:19.086 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-05-18T21:37:19.088 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-05-18T21:37:19.125 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.181) 0:02:35.174 ************ 2020-05-18T21:37:19.150 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-05-18T21:37:19.194 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.069) 0:02:35.243 ************ 2020-05-18T21:37:19.320 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-05-18T21:37:19.322 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-05-18T21:37:19.365 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.168) 0:02:35.412 ************ 2020-05-18T21:37:19.394 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-05-18T21:37:19.396 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.431 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.068) 0:02:35.480 ************ 2020-05-18T21:37:19.472 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.040) 0:02:35.520 ************ 2020-05-18T21:37:19.482 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.520 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.048) 0:02:35.569 ************ 2020-05-18T21:37:19.531 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-05-18T21:37:19.533 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi196.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.544 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.546 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.583 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.063) 0:02:35.632 ************ 2020-05-18T21:37:19.593 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.594 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.596 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.597 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi080.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.647 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.063) 0:02:35.695 ************ 2020-05-18T21:37:19.659 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-05-18T21:37:19.661 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.662 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.663 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.663 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.664 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.704 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.057) 0:02:35.753 ************ 2020-05-18T21:37:19.714 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.716 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.718 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.719 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.720 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.770 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.065) 0:02:35.819 ************ 2020-05-18T21:37:19.780 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-18T21:37:19.782 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.821 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.051) 0:02:35.870 ************ 2020-05-18T21:37:19.831 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.876 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.054) 0:02:35.925 ************ 2020-05-18T21:37:19.886 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-05-18T21:37:19.888 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.924 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.047) 0:02:35.973 ************ 2020-05-18T21:37:19.936 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:19.972 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:19 +0000 (0:00:00.047) 0:02:36.021 ************ 2020-05-18T21:37:19.982 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-05-18T21:37:19.984 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:20.021 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:20 +0000 (0:00:00.049) 0:02:36.070 ************ 2020-05-18T21:37:20.031 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:20.072 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:20 +0000 (0:00:00.050) 0:02:36.120 ************ 2020-05-18T21:37:20.082 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-05-18T21:37:20.084 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:20.120 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:20 +0000 (0:00:00.048) 0:02:36.169 ************ 2020-05-18T21:37:20.130 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:20.169 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:20 +0000 (0:00:00.048) 0:02:36.217 ************ 2020-05-18T21:37:20.179 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-05-18T21:37:20.181 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:20.219 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:20 +0000 (0:00:00.050) 0:02:36.268 ************ 2020-05-18T21:37:20.228 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:20.269 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:20 +0000 (0:00:00.049) 0:02:36.318 ************ 2020-05-18T21:37:20.278 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-05-18T21:37:20.280 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:20.319 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:20 +0000 (0:00:00.049) 0:02:36.368 ************ 2020-05-18T21:37:20.327 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:20.369 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:20 +0000 (0:00:00.050) 0:02:36.418 ************ 2020-05-18T21:37:20.377 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-05-18T21:37:20.379 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:20.420 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:20 +0000 (0:00:00.050) 0:02:36.468 ************ 2020-05-18T21:37:20.428 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:20.483 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:20 +0000 (0:00:00.063) 0:02:36.532 ************ 2020-05-18T21:37:20.493 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-05-18T21:37:20.495 INFO:teuthology.task.ansible.out:skipping: [smithi196.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:20.530 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:20 +0000 (0:00:00.047) 0:02:36.579 ************ 2020-05-18T21:37:20.538 INFO:teuthology.task.ansible.out:skipping: [smithi080.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-18T21:37:20.576 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:20 +0000 (0:00:00.045) 0:02:36.625 ************ 2020-05-18T21:37:20.738 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-05-18T21:37:20.740 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.010990", "end": "2020-05-18 21:37:21.136433", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-05-18 21:37:21.125443", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:20.791 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.009847", "end": "2020-05-18 21:37:20.369103", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-05-18 21:37:20.359256", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:20.914 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.005760", "end": "2020-05-18 21:37:21.305154", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-05-18 21:37:21.299394", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:20.976 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006197", "end": "2020-05-18 21:37:20.558013", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-05-18 21:37:20.551816", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:21.079 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005352", "end": "2020-05-18 21:37:21.480991", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-05-18 21:37:21.475639", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:21.152 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005675", "end": "2020-05-18 21:37:20.734453", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-05-18 21:37:20.728778", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:21.286 INFO:teuthology.task.ansible.out:ok: [smithi196.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.037800", "end": "2020-05-18 21:37:21.683015", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-05-18 21:37:21.645215", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:21.330 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:21 +0000 (0:00:00.753) 0:02:37.379 ************ 2020-05-18T21:37:21.350 INFO:teuthology.task.ansible.out:ok: [smithi080.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.027116", "end": "2020-05-18 21:37:20.927455", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-05-18 21:37:20.900339", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-18T21:37:21.395 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:21 +0000 (0:00:00.064) 0:02:37.443 ************ 2020-05-18T21:37:21.405 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-05-18T21:37:21.453 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:21 +0000 (0:00:00.057) 0:02:37.501 ************ 2020-05-18T21:37:21.501 INFO:teuthology.task.ansible.out:Monday 18 May 2020 21:37:21 +0000 (0:00:00.049) 0:02:37.550 ************ 2020-05-18T21:37:21.644 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-05-18T21:37:21.646 INFO:teuthology.task.ansible.out:changed: [smithi196.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-05-18T21:37:21.724 INFO:teuthology.task.ansible.out:changed: [smithi080.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-05-18T21:37:21.816 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2020-05-18T21:37:21.818 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-05-18T21:37:21.820 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2020-05-18T21:37:21.821 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi080.front.sepia.ceph.com : ok=114 changed=29 unreachable=0 failed=0 skipped=170 rescued=0 ignored=0 smithi196.front.sepia.ceph.com : ok=114 changed=29 unreachable=0 failed=0 skipped=170 rescued=0 ignored=0 Monday 18 May 2020 21:37:21 +0000 (0:00:00.318) 0:02:37.869 ************ =============================================================================== users : Create all admin users with sudo access. ----------------------- 17.88s users : Update authorized_keys using the keys repo --------------------- 16.57s testnode : Ensure packages are not present. ---------------------------- 12.27s testnode : Upgrade packages -------------------------------------------- 12.05s testnode : Install packages --------------------------------------------- 9.37s testnode : Zap all non-root disks --------------------------------------- 8.22s users : Remove revoked users -------------------------------------------- 6.07s testnode : Install apt keys --------------------------------------------- 2.57s testnode : Install packages via pip ------------------------------------- 2.42s common : Update apt cache ----------------------------------------------- 2.32s testnode : Create logical volume(s) ------------------------------------- 2.30s Gathering Facts --------------------------------------------------------- 2.22s testnode : Update apt cache. -------------------------------------------- 2.09s testnode : Blow away lingering OSD data and FSIDs ----------------------- 1.80s users : Update authorized_keys for each user with literal keys ---------- 1.54s Gathering Facts --------------------------------------------------------- 1.17s testnode : Set the system's hostname ------------------------------------ 1.08s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 1.02s testnode : Install python-pip on a 2020-05-18T21:37:21.823 INFO:teuthology.task.ansible.out:pt based systems. --------------------- 0.98s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 0.95s 2020-05-18T21:37:22.191 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi196.front.sepia.ceph.com'), Remote(name='ubuntu@smithi080.front.sepia.ceph.com')] 2020-05-18T21:37:22.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T21:37:22.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:37:22.239 INFO:teuthology.orchestra.run.smithi196:> true 2020-05-18T21:37:22.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T21:37:22.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T21:37:22.399 INFO:teuthology.orchestra.run.smithi080:> true 2020-05-18T21:37:22.520 INFO:teuthology.run_tasks:Running task clock... 2020-05-18T21:37:22.531 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-05-18T21:37:22.531 INFO:teuthology.orchestra.run:Running command with timeout 60 2020-05-18T21:37:22.533 INFO:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-05-18T21:37:22.569 INFO:teuthology.orchestra.run.smithi196.stdout:18 May 21:37:22 ntpd[8220]: ntpd 4.2.8p12@1.3728-o (1): Starting 2020-05-18T21:37:22.569 INFO:teuthology.orchestra.run.smithi196.stdout:18 May 21:37:22 ntpd[8220]: Command line: ntpd -gq 2020-05-18T21:37:22.570 INFO:teuthology.orchestra.run.smithi196.stdout:18 May 21:37:23 ntpd[8220]: proto: precision = 0.085 usec (-23) 2020-05-18T21:37:22.571 INFO:teuthology.orchestra.run.smithi196.stdout:18 May 21:37:23 ntpd[8220]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-05-18T21:37:22.571 INFO:teuthology.orchestra.run.smithi196.stdout:18 May 21:37:23 ntpd[8220]: restrict ::: KOD does nothing without LIMITED. 2020-05-18T21:37:22.571 INFO:teuthology.orchestra.run.smithi196.stdout:18 May 21:37:23 ntpd[8220]: Listen and drop on 0 v6wildcard [::]:123 2020-05-18T21:37:22.571 INFO:teuthology.orchestra.run.smithi196.stdout:18 May 21:37:23 ntpd[8220]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-05-18T21:37:22.572 INFO:teuthology.orchestra.run.smithi196.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-05-18T21:37:22.572 INFO:teuthology.orchestra.run.smithi196.stderr:restrict ::: KOD does nothing without LIMITED. 2020-05-18T21:37:22.573 INFO:teuthology.orchestra.run.smithi196.stdout:18 May 21:37:23 ntpd[8220]: Listen normally on 2 lo 127.0.0.1:123 2020-05-18T21:37:22.573 INFO:teuthology.orchestra.run.smithi196.stdout:18 May 21:37:23 ntpd[8220]: Listen normally on 3 enp3s0f1 172.21.15.196:123 2020-05-18T21:37:22.573 INFO:teuthology.orchestra.run.smithi196.stdout:18 May 21:37:23 ntpd[8220]: Listen normally on 4 lo [::1]:123 2020-05-18T21:37:22.573 INFO:teuthology.orchestra.run.smithi196.stdout:18 May 21:37:23 ntpd[8220]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f47%5]:123 2020-05-18T21:37:22.573 INFO:teuthology.orchestra.run.smithi196.stdout:18 May 21:37:23 ntpd[8220]: Listening on routing socket on fd #22 for interface updates 2020-05-18T21:37:23.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:23.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:23.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:23.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:23.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:23.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:23.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:23.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:23.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:23.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:23.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:23.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:23.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:23.574 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:23.574 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:23.574 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:23.574 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:23.574 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:24 ntpd[8220]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:25.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:25.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:25.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:25.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:25.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:25.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:25.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:25.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:25.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:25.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:25.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:25.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:25.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:25.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:25.574 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:25.574 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:25.574 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:25.574 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:26 ntpd[8220]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:26.570 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:27 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:26.570 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:27 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:26.570 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:27 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:26.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:27 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:26.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:27 ntpd[8220]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:26.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:27 ntpd[8220]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:27.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:27.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:27.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:27.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:27.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:27.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:27.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:27.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:27.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:27.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:27.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:27.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:27.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:27.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:27.573 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:27.574 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:27.574 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:27.574 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:28 ntpd[8220]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:28.570 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:29 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:28.570 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:29 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:28.570 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:29 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:28.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:29 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:28.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:29 ntpd[8220]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:28.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:29 ntpd[8220]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:29.570 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:30 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:29.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:30 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:29.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:30 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:29.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:30 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:29.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:30 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:29.571 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:30 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:29.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:30 ntpd[8220]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:29.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:30 ntpd[8220]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:29.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:30 ntpd[8220]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:29.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:30 ntpd[8220]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:29.572 INFO:teuthology.orchestra.run.smithi196.stderr:18 May 21:37:29 ntpd[8220]: can't open /var/log/ntpstats/loopstats.20200518: Permission denied 2020-05-18T21:37:29.573 INFO:teuthology.orchestra.run.smithi196.stdout:18 May 21:37:29 ntpd[8220]: ntpd: time set -0.431644 s 2020-05-18T21:37:29.573 INFO:teuthology.orchestra.run.smithi196.stdout:ntpd: time set -0.431644s 2020-05-18T21:37:29.650 INFO:teuthology.orchestra.run.smithi196.stdout: remote refid st t when poll reach delay offset jitter 2020-05-18T21:37:29.650 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================== 2020-05-18T21:37:29.650 INFO:teuthology.orchestra.run.smithi196.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-18T21:37:29.650 INFO:teuthology.orchestra.run.smithi196.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-18T21:37:29.651 INFO:teuthology.orchestra.run.smithi196.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-18T21:37:29.651 INFO:teuthology.orchestra.run.smithi196.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-18T21:37:29.652 INFO:teuthology.orchestra.run:Running command with timeout 60 2020-05-18T21:37:29.652 INFO:teuthology.orchestra.run.smithi080:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-05-18T21:37:29.687 INFO:teuthology.orchestra.run.smithi080.stdout:18 May 21:37:29 ntpd[8292]: ntpd 4.2.8p12@1.3728-o (1): Starting 2020-05-18T21:37:29.687 INFO:teuthology.orchestra.run.smithi080.stdout:18 May 21:37:29 ntpd[8292]: Command line: ntpd -gq 2020-05-18T21:37:29.688 INFO:teuthology.orchestra.run.smithi080.stdout:18 May 21:37:29 ntpd[8292]: proto: precision = 0.080 usec (-23) 2020-05-18T21:37:29.689 INFO:teuthology.orchestra.run.smithi080.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-05-18T21:37:29.689 INFO:teuthology.orchestra.run.smithi080.stderr:restrict ::: KOD does nothing without LIMITED. 2020-05-18T21:37:29.689 INFO:teuthology.orchestra.run.smithi080.stdout:18 May 21:37:29 ntpd[8292]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-05-18T21:37:29.689 INFO:teuthology.orchestra.run.smithi080.stdout:18 May 21:37:29 ntpd[8292]: restrict ::: KOD does nothing without LIMITED. 2020-05-18T21:37:29.690 INFO:teuthology.orchestra.run.smithi080.stdout:18 May 21:37:29 ntpd[8292]: Listen and drop on 0 v6wildcard [::]:123 2020-05-18T21:37:29.690 INFO:teuthology.orchestra.run.smithi080.stdout:18 May 21:37:29 ntpd[8292]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-05-18T21:37:29.690 INFO:teuthology.orchestra.run.smithi080.stdout:18 May 21:37:29 ntpd[8292]: Listen normally on 2 lo 127.0.0.1:123 2020-05-18T21:37:29.690 INFO:teuthology.orchestra.run.smithi080.stdout:18 May 21:37:29 ntpd[8292]: Listen normally on 3 enp3s0f1 172.21.15.80:123 2020-05-18T21:37:29.690 INFO:teuthology.orchestra.run.smithi080.stdout:18 May 21:37:29 ntpd[8292]: Listen normally on 4 lo [::1]:123 2020-05-18T21:37:29.691 INFO:teuthology.orchestra.run.smithi080.stdout:18 May 21:37:29 ntpd[8292]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cba9%5]:123 2020-05-18T21:37:29.691 INFO:teuthology.orchestra.run.smithi080.stdout:18 May 21:37:29 ntpd[8292]: Listening on routing socket on fd #22 for interface updates 2020-05-18T21:37:30.687 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:30 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:30.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:30 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:30.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:30 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:30.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:30 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:30.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:30 ntpd[8292]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:30.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:30 ntpd[8292]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:31.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:31 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:31.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:31 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:31.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:31 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:31.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:31 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:31.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:31 ntpd[8292]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:31.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:31 ntpd[8292]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:32.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:32.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:32.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:32.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:32.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:32.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:32.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:32.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:32.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:32.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:32.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:32.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:32.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:32.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:32.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:32.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:32.692 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:32.692 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:32 ntpd[8292]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:33.687 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:33 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:33.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:33 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:33.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:33 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:33.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:33 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:33.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:33 ntpd[8292]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:33.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:33 ntpd[8292]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:34.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:34.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:34.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:34.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:34.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:34.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:34.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:34.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:34.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:34.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:34.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:34.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:34.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:34.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:34.692 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:34.692 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:34.692 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:34.692 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:34 ntpd[8292]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:35.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:35 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:35.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:35 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:35.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:35 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:35.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:35 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:35.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:35 ntpd[8292]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:35.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:35 ntpd[8292]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:36.688 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:36.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:36.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:36.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:36.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:36.689 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:36.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:36.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:36.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:36.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:36.690 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-05-18T21:37:36.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: can't open /var/log/ntpstats/rawstats.20200518: Permission denied 2020-05-18T21:37:36.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-05-18T21:37:36.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: can't open /var/log/ntpstats/peerstats.20200518: Permission denied 2020-05-18T21:37:36.691 INFO:teuthology.orchestra.run.smithi080.stderr:18 May 21:37:36 ntpd[8292]: can't open /var/log/ntpstats/loopstats.20200518: Permission denied 2020-05-18T21:37:36.691 INFO:teuthology.orchestra.run.smithi080.stdout:18 May 21:37:36 ntpd[8292]: ntpd: time set +0.384607 s 2020-05-18T21:37:36.692 INFO:teuthology.orchestra.run.smithi080.stdout:ntpd: time set +0.384607s 2020-05-18T21:37:36.746 INFO:teuthology.orchestra.run.smithi080.stdout: remote refid st t when poll reach delay offset jitter 2020-05-18T21:37:36.746 INFO:teuthology.orchestra.run.smithi080.stdout:============================================================================== 2020-05-18T21:37:36.747 INFO:teuthology.orchestra.run.smithi080.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-18T21:37:36.747 INFO:teuthology.orchestra.run.smithi080.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-18T21:37:36.747 INFO:teuthology.orchestra.run.smithi080.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-18T21:37:36.748 INFO:teuthology.orchestra.run.smithi080.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-18T21:37:36.748 INFO:teuthology.run_tasks:Running task install... 2020-05-18T21:37:36.761 DEBUG:teuthology.task.install:project ceph 2020-05-18T21:37:36.761 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '543315c934420269aa12ef2f9dec2c9eadb4fa6f'}} 2020-05-18T21:37:36.762 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '543315c934420269aa12ef2f9dec2c9eadb4fa6f'} 2020-05-18T21:37:36.762 INFO:teuthology.task.install:Using flavor: basic 2020-05-18T21:37:36.770 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-05-18T21:37:36.770 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2020-05-18T21:37:36.771 INFO:teuthology.orchestra.run.smithi196:> sudo lsb_release -is 2020-05-18T21:37:36.824 INFO:teuthology.orchestra.run.smithi196.stdout:Ubuntu 2020-05-18T21:37:36.830 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-05-18T21:37:36.831 INFO:teuthology.orchestra.run.smithi080:> sudo lsb_release -is 2020-05-18T21:37:36.831 INFO:teuthology.orchestra.run.smithi196:> sudo apt-key list | grep Ceph 2020-05-18T21:37:36.882 INFO:teuthology.orchestra.run.smithi080.stdout:Ubuntu 2020-05-18T21:37:36.887 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-05-18T21:37:36.888 INFO:teuthology.orchestra.run.smithi080:> sudo apt-key list | grep Ceph 2020-05-18T21:37:36.931 INFO:teuthology.orchestra.run.smithi196.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-05-18T21:37:36.987 INFO:teuthology.orchestra.run.smithi080.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-05-18T21:37:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-05-18T21:37:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:uid [ unknown] Ceph.com (release key) 2020-05-18T21:37:36.989 INFO:teuthology.orchestra.run.smithi196:> sudo lsb_release -is 2020-05-18T21:37:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:Ubuntu 2020-05-18T21:37:37.042 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-05-18T21:37:37.043 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-05-18T21:37:37.043 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-05-18T21:37:37.043 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=543315c934420269aa12ef2f9dec2c9eadb4fa6f 2020-05-18T21:37:37.051 INFO:teuthology.orchestra.run.smithi080.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-05-18T21:37:37.051 INFO:teuthology.orchestra.run.smithi080.stdout:uid [ unknown] Ceph.com (release key) 2020-05-18T21:37:37.052 INFO:teuthology.orchestra.run.smithi080:> sudo lsb_release -is 2020-05-18T21:37:37.099 INFO:teuthology.orchestra.run.smithi080.stdout:Ubuntu 2020-05-18T21:37:37.104 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-05-18T21:37:37.104 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-05-18T21:37:37.104 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-05-18T21:37:37.105 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=543315c934420269aa12ef2f9dec2c9eadb4fa6f 2020-05-18T21:37:37.287 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default/ 2020-05-18T21:37:37.288 INFO:teuthology.task.install.deb:Package version is 16.0.0-1640-g543315c9344-1focal 2020-05-18T21:37:37.340 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default/ 2020-05-18T21:37:37.341 INFO:teuthology.task.install.deb:Package version is 16.0.0-1640-g543315c9344-1focal 2020-05-18T21:37:37.468 INFO:teuthology.orchestra.run.smithi196:> sudo sh -c 'cat > /etc/apt/sources.list.d/ceph.list' 2020-05-18T21:37:37.485 INFO:teuthology.orchestra.run.smithi196:> sudo apt-get update 2020-05-18T21:37:37.530 INFO:teuthology.orchestra.run.smithi080:> sudo sh -c 'cat > /etc/apt/sources.list.d/ceph.list' 2020-05-18T21:37:37.548 INFO:teuthology.orchestra.run.smithi080:> sudo apt-get update 2020-05-18T21:37:37.634 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2020-05-18T21:37:37.748 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2020-05-18T21:37:37.819 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2020-05-18T21:37:37.819 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2020-05-18T21:37:37.825 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2020-05-18T21:37:37.850 INFO:teuthology.orchestra.run.smithi196.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal InRelease 2020-05-18T21:37:37.888 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2020-05-18T21:37:37.905 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2020-05-18T21:37:37.924 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal Release [7,671 B] 2020-05-18T21:37:37.938 INFO:teuthology.orchestra.run.smithi080.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal InRelease 2020-05-18T21:37:37.971 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2020-05-18T21:37:37.994 INFO:teuthology.orchestra.run.smithi196.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal Release.gpg 2020-05-18T21:37:38.008 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal Release [7,671 B] 2020-05-18T21:37:38.067 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main i386 Packages [3,069 B] 2020-05-18T21:37:38.082 INFO:teuthology.orchestra.run.smithi080.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal Release.gpg 2020-05-18T21:37:38.157 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2020-05-18T21:37:38.160 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2020-05-18T21:37:38.253 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main i386 Packages [3,069 B] 2020-05-18T21:37:38.287 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 26.6 kB in 1s (41.9 kB/s) 2020-05-18T21:37:38.464 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 26.6 kB in 1s (35.1 kB/s) 2020-05-18T21:37:38.908 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2020-05-18T21:37:38.927 INFO:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-1640-g543315c9344-1focal cephadm=16.0.0-1640-g543315c9344-1focal ceph-mds=16.0.0-1640-g543315c9344-1focal ceph-mgr=16.0.0-1640-g543315c9344-1focal ceph-common=16.0.0-1640-g543315c9344-1focal ceph-fuse=16.0.0-1640-g543315c9344-1focal ceph-test=16.0.0-1640-g543315c9344-1focal radosgw=16.0.0-1640-g543315c9344-1focal python3-rados=16.0.0-1640-g543315c9344-1focal python3-rgw=16.0.0-1640-g543315c9344-1focal python3-cephfs=16.0.0-1640-g543315c9344-1focal python3-rbd=16.0.0-1640-g543315c9344-1focal libcephfs2=16.0.0-1640-g543315c9344-1focal libcephfs-dev=16.0.0-1640-g543315c9344-1focal librados2=16.0.0-1640-g543315c9344-1focal librbd1=16.0.0-1640-g543315c9344-1focal rbd-fuse=16.0.0-1640-g543315c9344-1focal python3-cephfs=16.0.0-1640-g543315c9344-1focal cephfs-shell=16.0.0-1640-g543315c9344-1focal 2020-05-18T21:37:38.994 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2020-05-18T21:37:39.082 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2020-05-18T21:37:39.102 INFO:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-1640-g543315c9344-1focal cephadm=16.0.0-1640-g543315c9344-1focal ceph-mds=16.0.0-1640-g543315c9344-1focal ceph-mgr=16.0.0-1640-g543315c9344-1focal ceph-common=16.0.0-1640-g543315c9344-1focal ceph-fuse=16.0.0-1640-g543315c9344-1focal ceph-test=16.0.0-1640-g543315c9344-1focal radosgw=16.0.0-1640-g543315c9344-1focal python3-rados=16.0.0-1640-g543315c9344-1focal python3-rgw=16.0.0-1640-g543315c9344-1focal python3-cephfs=16.0.0-1640-g543315c9344-1focal python3-rbd=16.0.0-1640-g543315c9344-1focal libcephfs2=16.0.0-1640-g543315c9344-1focal libcephfs-dev=16.0.0-1640-g543315c9344-1focal librados2=16.0.0-1640-g543315c9344-1focal librbd1=16.0.0-1640-g543315c9344-1focal rbd-fuse=16.0.0-1640-g543315c9344-1focal python3-cephfs=16.0.0-1640-g543315c9344-1focal cephfs-shell=16.0.0-1640-g543315c9344-1focal 2020-05-18T21:37:39.114 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2020-05-18T21:37:39.115 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2020-05-18T21:37:39.176 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2020-05-18T21:37:39.230 INFO:teuthology.orchestra.run.smithi196.stdout:The following package was automatically installed and is no longer required: 2020-05-18T21:37:39.231 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.71.0 2020-05-18T21:37:39.232 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove it. 2020-05-18T21:37:39.232 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2020-05-18T21:37:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-05-18T21:37:39.232 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-05-18T21:37:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore 2020-05-18T21:37:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: liblbfgsb0 libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 2020-05-18T21:37:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 librdkafka1 librgw2 python-pastedeploy-tpl 2020-05-18T21:37:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-bs4 2020-05-18T21:37:39.233 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2020-05-18T21:37:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-decorator 2020-05-18T21:37:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth python3-html5lib python3-joblib python3-json-pointer 2020-05-18T21:37:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch python3-kubernetes python3-logutils python3-lxml 2020-05-18T21:37:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako python3-markupsafe python3-openssl python3-packaging 2020-05-18T21:37:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2020-05-18T21:37:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2020-05-18T21:37:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2020-05-18T21:37:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2020-05-18T21:37:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy python3-simplegeneric python3-singledispatch python3-sklearn 2020-05-18T21:37:39.235 INFO:teuthology.orchestra.run.smithi196.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2020-05-18T21:37:39.236 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2020-05-18T21:37:39.236 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-05-18T21:37:39.237 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2020-05-18T21:37:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: python3-influxdb python-attr-doc python3-genshi python3-lxml-dbg 2020-05-18T21:37:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: python-lxml-doc python3-beaker python-mako-doc python-openssl-doc 2020-05-18T21:37:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: python3-openssl-dbg httpd-wsgi libapache2-mod-python libapache2-mod-scgi 2020-05-18T21:37:39.237 INFO:teuthology.orchestra.run.smithi196.stdout: libjs-mochikit python-pecan-doc python-psutil-doc subversion 2020-05-18T21:37:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: python-pyinotify-doc python-scipy-doc python3-dap python-sklearn-doc 2020-05-18T21:37:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: ipython3 python-waitress-doc python-webob-doc python-webtest-doc 2020-05-18T21:37:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: python-werkzeug-doc python3-termcolor python3-watchdog 2020-05-18T21:37:39.238 INFO:teuthology.orchestra.run.smithi196.stdout:Recommended packages: 2020-05-18T21:37:39.238 INFO:teuthology.orchestra.run.smithi196.stdout: btrfs-tools 2020-05-18T21:37:39.303 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2020-05-18T21:37:39.304 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2020-05-18T21:37:39.322 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2020-05-18T21:37:39.322 INFO:teuthology.orchestra.run.smithi196.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-05-18T21:37:39.323 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-05-18T21:37:39.323 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-05-18T21:37:39.323 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2020-05-18T21:37:39.323 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-dev libcephfs2 libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2020-05-18T21:37:39.323 INFO:teuthology.orchestra.run.smithi196.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2020-05-18T21:37:39.324 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka1 librgw2 python-pastedeploy-tpl python3-atomicwrites python3-attr 2020-05-18T21:37:39.325 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt python3-bs4 python3-cachetools python3-ceph-argparse 2020-05-18T21:37:39.325 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2020-05-18T21:37:39.325 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama python3-decorator python3-google-auth python3-html5lib 2020-05-18T21:37:39.325 INFO:teuthology.orchestra.run.smithi196.stdout: python3-joblib python3-json-pointer python3-jsonpatch python3-kubernetes 2020-05-18T21:37:39.325 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-05-18T21:37:39.326 INFO:teuthology.orchestra.run.smithi196.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2020-05-18T21:37:39.326 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2020-05-18T21:37:39.326 INFO:teuthology.orchestra.run.smithi196.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2020-05-18T21:37:39.326 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyinotify python3-pyperclip python3-pytest python3-rados python3-rbd 2020-05-18T21:37:39.326 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2020-05-18T21:37:39.326 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa python3-scipy python3-simplegeneric python3-singledispatch 2020-05-18T21:37:39.327 INFO:teuthology.orchestra.run.smithi196.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2020-05-18T21:37:39.327 INFO:teuthology.orchestra.run.smithi196.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2020-05-18T21:37:39.327 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket python3-webtest python3-werkzeug radosgw rbd-fuse socat 2020-05-18T21:37:39.327 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet 2020-05-18T21:37:39.327 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages will be upgraded: 2020-05-18T21:37:39.328 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 librbd1 2020-05-18T21:37:39.420 INFO:teuthology.orchestra.run.smithi080.stdout:The following package was automatically installed and is no longer required: 2020-05-18T21:37:39.421 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.71.0 2020-05-18T21:37:39.421 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove it. 2020-05-18T21:37:39.422 INFO:teuthology.orchestra.run.smithi080.stdout:The following additional packages will be installed: 2020-05-18T21:37:39.422 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-05-18T21:37:39.422 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-05-18T21:37:39.422 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore 2020-05-18T21:37:39.422 INFO:teuthology.orchestra.run.smithi080.stdout: liblbfgsb0 libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 2020-05-18T21:37:39.423 INFO:teuthology.orchestra.run.smithi080.stdout: libradosstriper1 librdkafka1 librgw2 python-pastedeploy-tpl 2020-05-18T21:37:39.423 INFO:teuthology.orchestra.run.smithi080.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-bs4 2020-05-18T21:37:39.423 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2020-05-18T21:37:39.423 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-decorator 2020-05-18T21:37:39.423 INFO:teuthology.orchestra.run.smithi080.stdout: python3-google-auth python3-html5lib python3-joblib python3-json-pointer 2020-05-18T21:37:39.424 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpatch python3-kubernetes python3-logutils python3-lxml 2020-05-18T21:37:39.424 INFO:teuthology.orchestra.run.smithi080.stdout: python3-mako python3-markupsafe python3-openssl python3-packaging 2020-05-18T21:37:39.424 INFO:teuthology.orchestra.run.smithi080.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2020-05-18T21:37:39.424 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2020-05-18T21:37:39.424 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2020-05-18T21:37:39.424 INFO:teuthology.orchestra.run.smithi080.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2020-05-18T21:37:39.425 INFO:teuthology.orchestra.run.smithi080.stdout: python3-scipy python3-simplegeneric python3-singledispatch python3-sklearn 2020-05-18T21:37:39.425 INFO:teuthology.orchestra.run.smithi080.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2020-05-18T21:37:39.425 INFO:teuthology.orchestra.run.smithi080.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2020-05-18T21:37:39.425 INFO:teuthology.orchestra.run.smithi080.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-05-18T21:37:39.426 INFO:teuthology.orchestra.run.smithi080.stdout:Suggested packages: 2020-05-18T21:37:39.426 INFO:teuthology.orchestra.run.smithi080.stdout: python3-influxdb python-attr-doc python3-genshi python3-lxml-dbg 2020-05-18T21:37:39.426 INFO:teuthology.orchestra.run.smithi080.stdout: python-lxml-doc python3-beaker python-mako-doc python-openssl-doc 2020-05-18T21:37:39.426 INFO:teuthology.orchestra.run.smithi080.stdout: python3-openssl-dbg httpd-wsgi libapache2-mod-python libapache2-mod-scgi 2020-05-18T21:37:39.427 INFO:teuthology.orchestra.run.smithi080.stdout: libjs-mochikit python-pecan-doc python-psutil-doc subversion 2020-05-18T21:37:39.427 INFO:teuthology.orchestra.run.smithi080.stdout: python-pyinotify-doc python-scipy-doc python3-dap python-sklearn-doc 2020-05-18T21:37:39.427 INFO:teuthology.orchestra.run.smithi080.stdout: ipython3 python-waitress-doc python-webob-doc python-webtest-doc 2020-05-18T21:37:39.427 INFO:teuthology.orchestra.run.smithi080.stdout: python-werkzeug-doc python3-termcolor python3-watchdog 2020-05-18T21:37:39.427 INFO:teuthology.orchestra.run.smithi080.stdout:Recommended packages: 2020-05-18T21:37:39.428 INFO:teuthology.orchestra.run.smithi080.stdout: btrfs-tools 2020-05-18T21:37:39.509 INFO:teuthology.orchestra.run.smithi080.stdout:The following NEW packages will be installed: 2020-05-18T21:37:39.509 INFO:teuthology.orchestra.run.smithi080.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-05-18T21:37:39.510 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-05-18T21:37:39.510 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-05-18T21:37:39.510 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq 2020-05-18T21:37:39.510 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs-dev libcephfs2 libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2020-05-18T21:37:39.510 INFO:teuthology.orchestra.run.smithi080.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2020-05-18T21:37:39.511 INFO:teuthology.orchestra.run.smithi080.stdout: librdkafka1 librgw2 python-pastedeploy-tpl python3-atomicwrites python3-attr 2020-05-18T21:37:39.512 INFO:teuthology.orchestra.run.smithi080.stdout: python3-bcrypt python3-bs4 python3-cachetools python3-ceph-argparse 2020-05-18T21:37:39.512 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2020-05-18T21:37:39.512 INFO:teuthology.orchestra.run.smithi080.stdout: python3-colorama python3-decorator python3-google-auth python3-html5lib 2020-05-18T21:37:39.512 INFO:teuthology.orchestra.run.smithi080.stdout: python3-joblib python3-json-pointer python3-jsonpatch python3-kubernetes 2020-05-18T21:37:39.512 INFO:teuthology.orchestra.run.smithi080.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-05-18T21:37:39.513 INFO:teuthology.orchestra.run.smithi080.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2020-05-18T21:37:39.513 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2020-05-18T21:37:39.513 INFO:teuthology.orchestra.run.smithi080.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2020-05-18T21:37:39.513 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyinotify python3-pyperclip python3-pytest python3-rados python3-rbd 2020-05-18T21:37:39.513 INFO:teuthology.orchestra.run.smithi080.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2020-05-18T21:37:39.514 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rsa python3-scipy python3-simplegeneric python3-singledispatch 2020-05-18T21:37:39.514 INFO:teuthology.orchestra.run.smithi080.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2020-05-18T21:37:39.514 INFO:teuthology.orchestra.run.smithi080.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2020-05-18T21:37:39.514 INFO:teuthology.orchestra.run.smithi080.stdout: python3-websocket python3-webtest python3-werkzeug radosgw rbd-fuse socat 2020-05-18T21:37:39.514 INFO:teuthology.orchestra.run.smithi080.stdout: xmlstarlet 2020-05-18T21:37:39.514 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages will be upgraded: 2020-05-18T21:37:39.515 INFO:teuthology.orchestra.run.smithi080.stdout: librados2 librbd1 2020-05-18T21:37:39.515 INFO:teuthology.orchestra.run.smithi196.stdout:2 upgraded, 96 newly installed, 0 to remove and 45 not upgraded. 2020-05-18T21:37:39.515 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 152 MB of archives. 2020-05-18T21:37:39.516 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 692 MB of additional disk space will be used. 2020-05-18T21:37:39.516 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2020-05-18T21:37:39.611 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2020-05-18T21:37:39.644 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2020-05-18T21:37:39.699 INFO:teuthology.orchestra.run.smithi080.stdout:2 upgraded, 96 newly installed, 0 to remove and 45 not upgraded. 2020-05-18T21:37:39.699 INFO:teuthology.orchestra.run.smithi080.stdout:Need to get 152 MB of archives. 2020-05-18T21:37:39.699 INFO:teuthology.orchestra.run.smithi080.stdout:After this operation, 692 MB of additional disk space will be used. 2020-05-18T21:37:39.700 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2020-05-18T21:37:39.751 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 16.0.0-1640-g543315c9344-1focal [1,678 kB] 2020-05-18T21:37:39.770 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2020-05-18T21:37:39.816 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2020-05-18T21:37:39.868 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2020-05-18T21:37:39.895 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2020-05-18T21:37:39.903 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2020-05-18T21:37:39.918 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2020-05-18T21:37:39.954 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 16.0.0-1640-g543315c9344-1focal [1,678 kB] 2020-05-18T21:37:39.996 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2020-05-18T21:37:40.018 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2020-05-18T21:37:40.041 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2020-05-18T21:37:40.046 INFO:teuthology.orchestra.run.smithi196.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2020-05-18T21:37:40.052 INFO:teuthology.orchestra.run.smithi196.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2020-05-18T21:37:40.056 INFO:teuthology.orchestra.run.smithi196.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2020-05-18T21:37:40.073 INFO:teuthology.orchestra.run.smithi196.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2020-05-18T21:37:40.074 INFO:teuthology.orchestra.run.smithi196.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2020-05-18T21:37:40.076 INFO:teuthology.orchestra.run.smithi196.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2020-05-18T21:37:40.081 INFO:teuthology.orchestra.run.smithi196.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2020-05-18T21:37:40.123 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2020-05-18T21:37:40.132 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2020-05-18T21:37:40.138 INFO:teuthology.orchestra.run.smithi196.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2020-05-18T21:37:40.138 INFO:teuthology.orchestra.run.smithi196.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2020-05-18T21:37:40.143 INFO:teuthology.orchestra.run.smithi196.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2020-05-18T21:37:40.144 INFO:teuthology.orchestra.run.smithi196.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2020-05-18T21:37:40.147 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2020-05-18T21:37:40.148 INFO:teuthology.orchestra.run.smithi196.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-waitress all 1.4.1-1 [80.6 kB] 2020-05-18T21:37:40.219 INFO:teuthology.orchestra.run.smithi196.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2020-05-18T21:37:40.220 INFO:teuthology.orchestra.run.smithi196.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2020-05-18T21:37:40.239 INFO:teuthology.orchestra.run.smithi196.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2020-05-18T21:37:40.240 INFO:teuthology.orchestra.run.smithi196.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2020-05-18T21:37:40.240 INFO:teuthology.orchestra.run.smithi196.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2020-05-18T21:37:40.248 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2020-05-18T21:37:40.270 INFO:teuthology.orchestra.run.smithi080.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2020-05-18T21:37:40.276 INFO:teuthology.orchestra.run.smithi080.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2020-05-18T21:37:40.281 INFO:teuthology.orchestra.run.smithi080.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2020-05-18T21:37:40.285 INFO:teuthology.orchestra.run.smithi080.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2020-05-18T21:37:40.300 INFO:teuthology.orchestra.run.smithi196.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2020-05-18T21:37:40.302 INFO:teuthology.orchestra.run.smithi080.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2020-05-18T21:37:40.303 INFO:teuthology.orchestra.run.smithi080.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2020-05-18T21:37:40.304 INFO:teuthology.orchestra.run.smithi196.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2020-05-18T21:37:40.304 INFO:teuthology.orchestra.run.smithi080.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2020-05-18T21:37:40.310 INFO:teuthology.orchestra.run.smithi080.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2020-05-18T21:37:40.311 INFO:teuthology.orchestra.run.smithi196.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2020-05-18T21:37:40.312 INFO:teuthology.orchestra.run.smithi196.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-underscore all 1.9.1~dfsg-1 [98.6 kB] 2020-05-18T21:37:40.316 INFO:teuthology.orchestra.run.smithi196.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2020-05-18T21:37:40.367 INFO:teuthology.orchestra.run.smithi080.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2020-05-18T21:37:40.368 INFO:teuthology.orchestra.run.smithi080.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2020-05-18T21:37:40.372 INFO:teuthology.orchestra.run.smithi080.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2020-05-18T21:37:40.373 INFO:teuthology.orchestra.run.smithi080.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2020-05-18T21:37:40.377 INFO:teuthology.orchestra.run.smithi080.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-waitress all 1.4.1-1 [80.6 kB] 2020-05-18T21:37:40.381 INFO:teuthology.orchestra.run.smithi196.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2020-05-18T21:37:40.385 INFO:teuthology.orchestra.run.smithi196.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2020-05-18T21:37:40.386 INFO:teuthology.orchestra.run.smithi196.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2020-05-18T21:37:40.387 INFO:teuthology.orchestra.run.smithi196.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2020-05-18T21:37:40.389 INFO:teuthology.orchestra.run.smithi196.stdout:Get:37 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 16.0.0-1640-g543315c9344-1focal [3,229 kB] 2020-05-18T21:37:40.447 INFO:teuthology.orchestra.run.smithi080.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2020-05-18T21:37:40.448 INFO:teuthology.orchestra.run.smithi080.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2020-05-18T21:37:40.468 INFO:teuthology.orchestra.run.smithi080.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2020-05-18T21:37:40.469 INFO:teuthology.orchestra.run.smithi080.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2020-05-18T21:37:40.469 INFO:teuthology.orchestra.run.smithi080.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2020-05-18T21:37:40.528 INFO:teuthology.orchestra.run.smithi080.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2020-05-18T21:37:40.532 INFO:teuthology.orchestra.run.smithi080.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2020-05-18T21:37:40.540 INFO:teuthology.orchestra.run.smithi080.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2020-05-18T21:37:40.540 INFO:teuthology.orchestra.run.smithi080.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-underscore all 1.9.1~dfsg-1 [98.6 kB] 2020-05-18T21:37:40.545 INFO:teuthology.orchestra.run.smithi080.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2020-05-18T21:37:40.609 INFO:teuthology.orchestra.run.smithi080.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2020-05-18T21:37:40.614 INFO:teuthology.orchestra.run.smithi080.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2020-05-18T21:37:40.614 INFO:teuthology.orchestra.run.smithi080.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2020-05-18T21:37:40.615 INFO:teuthology.orchestra.run.smithi080.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2020-05-18T21:37:40.736 INFO:teuthology.orchestra.run.smithi196.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2020-05-18T21:37:40.795 INFO:teuthology.orchestra.run.smithi196.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2020-05-18T21:37:40.812 INFO:teuthology.orchestra.run.smithi196.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2020-05-18T21:37:40.863 INFO:teuthology.orchestra.run.smithi196.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2020-05-18T21:37:40.864 INFO:teuthology.orchestra.run.smithi196.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2020-05-18T21:37:40.864 INFO:teuthology.orchestra.run.smithi196.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2020-05-18T21:37:40.865 INFO:teuthology.orchestra.run.smithi196.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2020-05-18T21:37:40.866 INFO:teuthology.orchestra.run.smithi196.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2020-05-18T21:37:40.867 INFO:teuthology.orchestra.run.smithi196.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2020-05-18T21:37:40.867 INFO:teuthology.orchestra.run.smithi196.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2020-05-18T21:37:40.899 INFO:teuthology.orchestra.run.smithi196.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2020-05-18T21:37:40.904 INFO:teuthology.orchestra.run.smithi196.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-json-pointer all 2.0-0ubuntu1 [8,320 B] 2020-05-18T21:37:40.969 INFO:teuthology.orchestra.run.smithi080.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2020-05-18T21:37:40.980 INFO:teuthology.orchestra.run.smithi196.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jsonpatch all 1.23-3 [12.0 kB] 2020-05-18T21:37:40.981 INFO:teuthology.orchestra.run.smithi196.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2020-05-18T21:37:40.984 INFO:teuthology.orchestra.run.smithi196.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjq1 amd64 1.6-1 [121 kB] 2020-05-18T21:37:40.986 INFO:teuthology.orchestra.run.smithi196.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 jq amd64 1.6-1 [50.2 kB] 2020-05-18T21:37:40.988 INFO:teuthology.orchestra.run.smithi196.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2020-05-18T21:37:40.994 INFO:teuthology.orchestra.run.smithi196.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2020-05-18T21:37:40.996 INFO:teuthology.orchestra.run.smithi196.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2020-05-18T21:37:40.996 INFO:teuthology.orchestra.run.smithi196.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2020-05-18T21:37:41.023 INFO:teuthology.orchestra.run.smithi080.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2020-05-18T21:37:41.040 INFO:teuthology.orchestra.run.smithi196.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 16.0.0-1640-g543315c9344-1focal [460 kB] 2020-05-18T21:37:41.046 INFO:teuthology.orchestra.run.smithi080.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2020-05-18T21:37:41.061 INFO:teuthology.orchestra.run.smithi196.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2020-05-18T21:37:41.062 INFO:teuthology.orchestra.run.smithi196.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2020-05-18T21:37:41.077 INFO:teuthology.orchestra.run.smithi196.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2020-05-18T21:37:41.080 INFO:teuthology.orchestra.run.smithi196.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-lxml amd64 4.5.0-1 [1,381 kB] 2020-05-18T21:37:41.094 INFO:teuthology.orchestra.run.smithi196.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2020-05-18T21:37:41.094 INFO:teuthology.orchestra.run.smithi196.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2020-05-18T21:37:41.095 INFO:teuthology.orchestra.run.smithi196.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2020-05-18T21:37:41.095 INFO:teuthology.orchestra.run.smithi196.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2020-05-18T21:37:41.096 INFO:teuthology.orchestra.run.smithi080.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2020-05-18T21:37:41.096 INFO:teuthology.orchestra.run.smithi080.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2020-05-18T21:37:41.096 INFO:teuthology.orchestra.run.smithi196.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-py all 1.8.1-1 [65.4 kB] 2020-05-18T21:37:41.097 INFO:teuthology.orchestra.run.smithi080.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2020-05-18T21:37:41.097 INFO:teuthology.orchestra.run.smithi080.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2020-05-18T21:37:41.098 INFO:teuthology.orchestra.run.smithi080.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2020-05-18T21:37:41.099 INFO:teuthology.orchestra.run.smithi080.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2020-05-18T21:37:41.100 INFO:teuthology.orchestra.run.smithi080.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2020-05-18T21:37:41.111 INFO:teuthology.orchestra.run.smithi196.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 16.0.0-1640-g543315c9344-1focal [367 kB] 2020-05-18T21:37:41.132 INFO:teuthology.orchestra.run.smithi080.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2020-05-18T21:37:41.137 INFO:teuthology.orchestra.run.smithi080.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-json-pointer all 2.0-0ubuntu1 [8,320 B] 2020-05-18T21:37:41.142 INFO:teuthology.orchestra.run.smithi196.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2020-05-18T21:37:41.186 INFO:teuthology.orchestra.run.smithi196.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 16.0.0-1640-g543315c9344-1focal [20.6 kB] 2020-05-18T21:37:41.187 INFO:teuthology.orchestra.run.smithi196.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 16.0.0-1640-g543315c9344-1focal [153 kB] 2020-05-18T21:37:41.191 INFO:teuthology.orchestra.run.smithi196.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 16.0.0-1640-g543315c9344-1focal [24.2 kB] 2020-05-18T21:37:41.215 INFO:teuthology.orchestra.run.smithi080.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jsonpatch all 1.23-3 [12.0 kB] 2020-05-18T21:37:41.215 INFO:teuthology.orchestra.run.smithi080.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2020-05-18T21:37:41.216 INFO:teuthology.orchestra.run.smithi080.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjq1 amd64 1.6-1 [121 kB] 2020-05-18T21:37:41.219 INFO:teuthology.orchestra.run.smithi080.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 jq amd64 1.6-1 [50.2 kB] 2020-05-18T21:37:41.220 INFO:teuthology.orchestra.run.smithi080.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2020-05-18T21:37:41.222 INFO:teuthology.orchestra.run.smithi196.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2020-05-18T21:37:41.226 INFO:teuthology.orchestra.run.smithi080.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2020-05-18T21:37:41.228 INFO:teuthology.orchestra.run.smithi080.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2020-05-18T21:37:41.228 INFO:teuthology.orchestra.run.smithi080.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2020-05-18T21:37:41.230 INFO:teuthology.orchestra.run.smithi196.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 16.0.0-1640-g543315c9344-1focal [345 kB] 2020-05-18T21:37:41.266 INFO:teuthology.orchestra.run.smithi196.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 16.0.0-1640-g543315c9344-1focal [3,122 kB] 2020-05-18T21:37:41.293 INFO:teuthology.orchestra.run.smithi080.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2020-05-18T21:37:41.294 INFO:teuthology.orchestra.run.smithi080.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2020-05-18T21:37:41.310 INFO:teuthology.orchestra.run.smithi080.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2020-05-18T21:37:41.311 INFO:teuthology.orchestra.run.smithi080.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-lxml amd64 4.5.0-1 [1,381 kB] 2020-05-18T21:37:41.325 INFO:teuthology.orchestra.run.smithi080.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2020-05-18T21:37:41.325 INFO:teuthology.orchestra.run.smithi080.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2020-05-18T21:37:41.325 INFO:teuthology.orchestra.run.smithi080.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2020-05-18T21:37:41.326 INFO:teuthology.orchestra.run.smithi080.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2020-05-18T21:37:41.327 INFO:teuthology.orchestra.run.smithi080.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-py all 1.8.1-1 [65.4 kB] 2020-05-18T21:37:41.374 INFO:teuthology.orchestra.run.smithi080.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2020-05-18T21:37:41.455 INFO:teuthology.orchestra.run.smithi080.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2020-05-18T21:37:41.883 INFO:teuthology.orchestra.run.smithi196.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 16.0.0-1640-g543315c9344-1focal [115 kB] 2020-05-18T21:37:41.950 INFO:teuthology.orchestra.run.smithi196.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 16.0.0-1640-g543315c9344-1focal [355 kB] 2020-05-18T21:37:41.961 INFO:teuthology.orchestra.run.smithi196.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 16.0.0-1640-g543315c9344-1focal [20.0 MB] 2020-05-18T21:37:41.996 INFO:teuthology.orchestra.run.smithi080.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 16.0.0-1640-g543315c9344-1focal [3,229 kB] 2020-05-18T21:37:45.154 INFO:teuthology.orchestra.run.smithi080.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 16.0.0-1640-g543315c9344-1focal [460 kB] 2020-05-18T21:37:45.366 INFO:teuthology.orchestra.run.smithi080.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 16.0.0-1640-g543315c9344-1focal [367 kB] 2020-05-18T21:37:45.492 INFO:teuthology.orchestra.run.smithi196.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 16.0.0-1640-g543315c9344-1focal [5,367 kB] 2020-05-18T21:37:45.500 INFO:teuthology.orchestra.run.smithi080.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 16.0.0-1640-g543315c9344-1focal [20.6 kB] 2020-05-18T21:37:45.510 INFO:teuthology.orchestra.run.smithi080.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 16.0.0-1640-g543315c9344-1focal [153 kB] 2020-05-18T21:37:45.561 INFO:teuthology.orchestra.run.smithi080.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 16.0.0-1640-g543315c9344-1focal [24.2 kB] 2020-05-18T21:37:45.576 INFO:teuthology.orchestra.run.smithi080.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 16.0.0-1640-g543315c9344-1focal [345 kB] 2020-05-18T21:37:45.714 INFO:teuthology.orchestra.run.smithi080.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 16.0.0-1640-g543315c9344-1focal [3,122 kB] 2020-05-18T21:37:45.915 INFO:teuthology.orchestra.run.smithi196.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 16.0.0-1640-g543315c9344-1focal [140 kB] 2020-05-18T21:37:45.918 INFO:teuthology.orchestra.run.smithi196.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 16.0.0-1640-g543315c9344-1focal [1,373 kB] 2020-05-18T21:37:46.048 INFO:teuthology.orchestra.run.smithi196.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 16.0.0-1640-g543315c9344-1focal [6,199 kB] 2020-05-18T21:37:46.621 INFO:teuthology.orchestra.run.smithi196.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 16.0.0-1640-g543315c9344-1focal [24.0 MB] 2020-05-18T21:37:46.894 INFO:teuthology.orchestra.run.smithi080.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 16.0.0-1640-g543315c9344-1focal [115 kB] 2020-05-18T21:37:46.922 INFO:teuthology.orchestra.run.smithi080.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 16.0.0-1640-g543315c9344-1focal [355 kB] 2020-05-18T21:37:47.049 INFO:teuthology.orchestra.run.smithi080.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 16.0.0-1640-g543315c9344-1focal [20.0 MB] 2020-05-18T21:37:48.836 INFO:teuthology.orchestra.run.smithi196.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 16.0.0-1640-g543315c9344-1focal [3,808 B] 2020-05-18T21:37:48.837 INFO:teuthology.orchestra.run.smithi196.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 16.0.0-1640-g543315c9344-1focal [533 kB] 2020-05-18T21:37:48.847 INFO:teuthology.orchestra.run.smithi196.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 16.0.0-1640-g543315c9344-1focal [1,998 kB] 2020-05-18T21:37:48.982 INFO:teuthology.orchestra.run.smithi196.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 16.0.0-1640-g543315c9344-1focal [41.0 kB] 2020-05-18T21:37:48.983 INFO:teuthology.orchestra.run.smithi196.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 16.0.0-1640-g543315c9344-1focal [37.3 kB] 2020-05-18T21:37:48.985 INFO:teuthology.orchestra.run.smithi196.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 16.0.0-1640-g543315c9344-1focal [3,189 kB] 2020-05-18T21:37:49.204 INFO:teuthology.orchestra.run.smithi196.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 16.0.0-1640-g543315c9344-1focal [34.2 kB] 2020-05-18T21:37:49.230 INFO:teuthology.orchestra.run.smithi196.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 16.0.0-1640-g543315c9344-1focal [7,894 kB] 2020-05-18T21:37:49.805 INFO:teuthology.orchestra.run.smithi196.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 16.0.0-1640-g543315c9344-1focal [3,924 B] 2020-05-18T21:37:49.806 INFO:teuthology.orchestra.run.smithi196.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-rook all 16.0.0-1640-g543315c9344-1focal [22.3 kB] 2020-05-18T21:37:49.852 INFO:teuthology.orchestra.run.smithi196.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 16.0.0-1640-g543315c9344-1focal [40.0 MB] 2020-05-18T21:37:51.255 INFO:teuthology.orchestra.run.smithi080.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 16.0.0-1640-g543315c9344-1focal [5,367 kB] 2020-05-18T21:37:51.901 INFO:teuthology.orchestra.run.smithi080.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 16.0.0-1640-g543315c9344-1focal [140 kB] 2020-05-18T21:37:51.933 INFO:teuthology.orchestra.run.smithi080.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 16.0.0-1640-g543315c9344-1focal [1,373 kB] 2020-05-18T21:37:52.108 INFO:teuthology.orchestra.run.smithi080.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 16.0.0-1640-g543315c9344-1focal [6,199 kB] 2020-05-18T21:37:52.696 INFO:teuthology.orchestra.run.smithi196.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 16.0.0-1640-g543315c9344-1focal [15.7 kB] 2020-05-18T21:37:52.697 INFO:teuthology.orchestra.run.smithi196.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 16.0.0-1640-g543315c9344-1focal [17.9 kB] 2020-05-18T21:37:52.697 INFO:teuthology.orchestra.run.smithi196.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 16.0.0-1640-g543315c9344-1focal [9,601 kB] 2020-05-18T21:37:52.828 INFO:teuthology.orchestra.run.smithi080.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 16.0.0-1640-g543315c9344-1focal [24.0 MB] 2020-05-18T21:37:53.174 INFO:teuthology.orchestra.run.smithi196.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 16.0.0-1640-g543315c9344-1focal [59.7 kB] 2020-05-18T21:37:53.825 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 152 MB in 14s (11.0 MB/s) 2020-05-18T21:37:53.981 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyperclip. 2020-05-18T21:37:54.042 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 149536 files and directories currently installed.) 2020-05-18T21:37:54.046 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../00-python3-pyperclip_1.7.0-1_all.deb ... 2020-05-18T21:37:54.087 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2020-05-18T21:37:54.423 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-wcwidth. 2020-05-18T21:37:54.441 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../01-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2020-05-18T21:37:54.481 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2020-05-18T21:37:54.784 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cmd2. 2020-05-18T21:37:54.801 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../02-python3-cmd2_0.8.5-2.1_all.deb ... 2020-05-18T21:37:54.842 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2020-05-18T21:37:55.154 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-05-18T21:37:55.172 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../03-liblttng-ust0_2.11.0-1_amd64.deb ... 2020-05-18T21:37:55.254 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2020-05-18T21:37:55.477 INFO:teuthology.orchestra.run.smithi080.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 16.0.0-1640-g543315c9344-1focal [3,808 B] 2020-05-18T21:37:55.478 INFO:teuthology.orchestra.run.smithi080.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 16.0.0-1640-g543315c9344-1focal [533 kB] 2020-05-18T21:37:55.488 INFO:teuthology.orchestra.run.smithi080.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 16.0.0-1640-g543315c9344-1focal [1,998 kB] 2020-05-18T21:37:55.600 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../04-librbd1_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:37:55.692 INFO:teuthology.orchestra.run.smithi080.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 16.0.0-1640-g543315c9344-1focal [41.0 kB] 2020-05-18T21:37:55.693 INFO:teuthology.orchestra.run.smithi080.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 16.0.0-1640-g543315c9344-1focal [37.3 kB] 2020-05-18T21:37:55.694 INFO:teuthology.orchestra.run.smithi080.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 16.0.0-1640-g543315c9344-1focal [3,189 kB] 2020-05-18T21:37:55.724 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librbd1 (16.0.0-1640-g543315c9344-1focal) over (15.2.1-0ubuntu2) ... 2020-05-18T21:37:55.995 INFO:teuthology.orchestra.run.smithi080.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 16.0.0-1640-g543315c9344-1focal [34.2 kB] 2020-05-18T21:37:56.012 INFO:teuthology.orchestra.run.smithi080.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 16.0.0-1640-g543315c9344-1focal [7,894 kB] 2020-05-18T21:37:56.297 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../05-librados2_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:37:56.414 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librados2 (16.0.0-1640-g543315c9344-1focal) over (15.2.1-0ubuntu2) ... 2020-05-18T21:37:56.745 INFO:teuthology.orchestra.run.smithi080.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 16.0.0-1640-g543315c9344-1focal [3,924 B] 2020-05-18T21:37:56.745 INFO:teuthology.orchestra.run.smithi080.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-rook all 16.0.0-1640-g543315c9344-1focal [22.3 kB] 2020-05-18T21:37:56.746 INFO:teuthology.orchestra.run.smithi080.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 16.0.0-1640-g543315c9344-1focal [40.0 MB] 2020-05-18T21:37:57.101 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libcephfs2. 2020-05-18T21:37:57.118 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../06-libcephfs2_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:37:57.159 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libcephfs2 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:37:57.461 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rados. 2020-05-18T21:37:57.478 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../07-python3-rados_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:37:57.511 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rados (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:37:57.873 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-05-18T21:37:57.891 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../08-python3-ceph-argparse_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:37:57.922 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-ceph-argparse (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:37:58.225 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cephfs. 2020-05-18T21:37:58.244 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../09-python3-cephfs_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:37:58.283 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cephfs (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:37:58.611 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-ceph-common. 2020-05-18T21:37:58.631 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../10-python3-ceph-common_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:37:58.669 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-ceph-common (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:37:58.972 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-prettytable. 2020-05-18T21:37:58.990 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../11-python3-prettytable_0.7.2-5_all.deb ... 2020-05-18T21:37:59.030 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2020-05-18T21:37:59.233 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rbd. 2020-05-18T21:37:59.249 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../12-python3-rbd_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:37:59.283 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rbd (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:37:59.694 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-05-18T21:37:59.711 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../13-librabbitmq4_0.10.0-1_amd64.deb ... 2020-05-18T21:37:59.752 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2020-05-18T21:37:59.909 INFO:teuthology.orchestra.run.smithi080.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 16.0.0-1640-g543315c9344-1focal [15.7 kB] 2020-05-18T21:37:59.909 INFO:teuthology.orchestra.run.smithi080.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 16.0.0-1640-g543315c9344-1focal [17.9 kB] 2020-05-18T21:37:59.910 INFO:teuthology.orchestra.run.smithi080.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 16.0.0-1640-g543315c9344-1focal [9,601 kB] 2020-05-18T21:38:00.394 INFO:teuthology.orchestra.run.smithi080.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20200516.030215/543315c934420269aa12ef2f9dec2c9eadb4fa6f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 16.0.0-1640-g543315c9344-1focal [59.7 kB] 2020-05-18T21:38:00.614 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-05-18T21:38:00.634 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../14-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2020-05-18T21:38:00.671 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2020-05-18T21:38:00.976 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librgw2. 2020-05-18T21:38:00.995 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../15-librgw2_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:01.024 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librgw2 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:01.051 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 152 MB in 21s (7,265 kB/s) 2020-05-18T21:38:01.201 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyperclip. 2020-05-18T21:38:01.261 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 149536 files and directories currently installed.) 2020-05-18T21:38:01.266 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../00-python3-pyperclip_1.7.0-1_all.deb ... 2020-05-18T21:38:01.307 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2020-05-18T21:38:01.552 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rgw. 2020-05-18T21:38:01.570 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../16-python3-rgw_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:01.610 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rgw (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:01.611 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-wcwidth. 2020-05-18T21:38:01.627 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../01-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2020-05-18T21:38:01.669 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2020-05-18T21:38:01.929 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libleveldb1d:amd64. 2020-05-18T21:38:01.948 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../17-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2020-05-18T21:38:01.955 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cmd2. 2020-05-18T21:38:01.975 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../02-python3-cmd2_0.8.5-2.1_all.deb ... 2020-05-18T21:38:01.987 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2020-05-18T21:38:02.013 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2020-05-18T21:38:02.317 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-05-18T21:38:02.336 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../03-liblttng-ust0_2.11.0-1_amd64.deb ... 2020-05-18T21:38:02.366 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liboath0. 2020-05-18T21:38:02.385 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../18-liboath0_2.6.1-1.3_amd64.deb ... 2020-05-18T21:38:02.409 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2020-05-18T21:38:02.424 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2020-05-18T21:38:02.668 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libradosstriper1. 2020-05-18T21:38:02.687 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../19-libradosstriper1_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:02.718 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libradosstriper1 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:02.746 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../04-librbd1_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:02.871 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librbd1 (16.0.0-1640-g543315c9344-1focal) over (15.2.1-0ubuntu2) ... 2020-05-18T21:38:03.163 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-common. 2020-05-18T21:38:03.183 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../20-ceph-common_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:03.221 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-common (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:03.412 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../05-librados2_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:03.542 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librados2 (16.0.0-1640-g543315c9344-1focal) over (15.2.1-0ubuntu2) ... 2020-05-18T21:38:04.205 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libcephfs2. 2020-05-18T21:38:04.221 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../06-libcephfs2_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:04.263 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libcephfs2 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:04.557 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rados. 2020-05-18T21:38:04.575 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../07-python3-rados_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:04.615 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rados (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:04.902 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-05-18T21:38:04.918 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../08-python3-ceph-argparse_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:04.951 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-ceph-argparse (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:05.260 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-base. 2020-05-18T21:38:05.271 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cephfs. 2020-05-18T21:38:05.279 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../21-ceph-base_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:05.291 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../09-python3-cephfs_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:05.328 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-base (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:05.329 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cephfs (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:05.657 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-ceph-common. 2020-05-18T21:38:05.674 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../10-python3-ceph-common_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:05.716 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-ceph-common (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:06.010 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-prettytable. 2020-05-18T21:38:06.030 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../11-python3-prettytable_0.7.2-5_all.deb ... 2020-05-18T21:38:06.069 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2020-05-18T21:38:06.231 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-openssl. 2020-05-18T21:38:06.248 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../22-python3-openssl_19.0.0-1build1_all.deb ... 2020-05-18T21:38:06.289 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2020-05-18T21:38:06.305 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rbd. 2020-05-18T21:38:06.324 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../12-python3-rbd_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:06.363 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rbd (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:06.600 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-05-18T21:38:06.617 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../23-ceph-mgr-modules-core_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:06.658 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-modules-core (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:06.725 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-05-18T21:38:06.744 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../13-librabbitmq4_0.10.0-1_amd64.deb ... 2020-05-18T21:38:06.782 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2020-05-18T21:38:07.021 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-bcrypt. 2020-05-18T21:38:07.039 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../24-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2020-05-18T21:38:07.079 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2020-05-18T21:38:07.159 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-05-18T21:38:07.179 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../14-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2020-05-18T21:38:07.227 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2020-05-18T21:38:07.399 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cherrypy3. 2020-05-18T21:38:07.416 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../25-python3-cherrypy3_8.9.1-6_all.deb ... 2020-05-18T21:38:07.456 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2020-05-18T21:38:07.514 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librgw2. 2020-05-18T21:38:07.530 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../15-librgw2_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:07.563 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librgw2 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:07.810 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-logutils. 2020-05-18T21:38:07.828 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../26-python3-logutils_0.3.3-7_all.deb ... 2020-05-18T21:38:07.869 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-logutils (0.3.3-7) ... 2020-05-18T21:38:08.133 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rgw. 2020-05-18T21:38:08.150 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../16-python3-rgw_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:08.179 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-markupsafe. 2020-05-18T21:38:08.196 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../27-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2020-05-18T21:38:08.208 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rgw (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:08.237 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2020-05-18T21:38:08.519 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libleveldb1d:amd64. 2020-05-18T21:38:08.535 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../17-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2020-05-18T21:38:08.549 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-mako. 2020-05-18T21:38:08.567 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../28-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2020-05-18T21:38:08.577 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2020-05-18T21:38:08.607 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2020-05-18T21:38:08.931 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liboath0. 2020-05-18T21:38:08.947 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../18-liboath0_2.6.1-1.3_amd64.deb ... 2020-05-18T21:38:08.960 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-simplegeneric. 2020-05-18T21:38:08.979 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../29-python3-simplegeneric_0.8.1-3_all.deb ... 2020-05-18T21:38:08.989 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2020-05-18T21:38:09.043 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2020-05-18T21:38:09.267 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libradosstriper1. 2020-05-18T21:38:09.283 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../19-libradosstriper1_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:09.316 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libradosstriper1 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:09.346 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-singledispatch. 2020-05-18T21:38:09.364 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../30-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2020-05-18T21:38:09.405 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2020-05-18T21:38:09.690 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-webob. 2020-05-18T21:38:09.710 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../31-python3-webob_1%3a1.8.5-2_all.deb ... 2020-05-18T21:38:09.748 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2020-05-18T21:38:09.787 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-common. 2020-05-18T21:38:09.803 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../20-ceph-common_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:09.845 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-common (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:10.102 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-soupsieve. 2020-05-18T21:38:10.118 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../32-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2020-05-18T21:38:10.160 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2020-05-18T21:38:10.446 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-bs4. 2020-05-18T21:38:10.466 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../33-python3-bs4_4.8.2-1_all.deb ... 2020-05-18T21:38:10.504 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2020-05-18T21:38:11.466 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-waitress. 2020-05-18T21:38:11.484 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../34-python3-waitress_1.4.1-1_all.deb ... 2020-05-18T21:38:11.527 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-waitress (1.4.1-1) ... 2020-05-18T21:38:11.935 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-tempita. 2020-05-18T21:38:11.953 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../35-python3-tempita_0.5.2-6_all.deb ... 2020-05-18T21:38:11.993 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-tempita (0.5.2-6) ... 2020-05-18T21:38:12.109 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-base. 2020-05-18T21:38:12.128 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../21-ceph-base_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:12.170 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-base (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:12.296 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-paste. 2020-05-18T21:38:12.313 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../36-python3-paste_3.4.0+dfsg1-1_all.deb ... 2020-05-18T21:38:12.354 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2020-05-18T21:38:12.691 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-05-18T21:38:12.708 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../37-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2020-05-18T21:38:12.758 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2020-05-18T21:38:13.060 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pastedeploy. 2020-05-18T21:38:13.077 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../38-python3-pastedeploy_2.1.0-1_all.deb ... 2020-05-18T21:38:13.081 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-openssl. 2020-05-18T21:38:13.097 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../22-python3-openssl_19.0.0-1build1_all.deb ... 2020-05-18T21:38:13.118 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2020-05-18T21:38:13.131 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2020-05-18T21:38:13.425 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-05-18T21:38:13.444 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../23-ceph-mgr-modules-core_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:13.447 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-webtest. 2020-05-18T21:38:13.466 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../39-python3-webtest_2.0.32-2_all.deb ... 2020-05-18T21:38:13.483 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-modules-core (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:13.504 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-webtest (2.0.32-2) ... 2020-05-18T21:38:13.816 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pecan. 2020-05-18T21:38:13.834 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../40-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2020-05-18T21:38:13.847 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-bcrypt. 2020-05-18T21:38:13.865 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../24-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2020-05-18T21:38:13.874 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2020-05-18T21:38:13.905 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2020-05-18T21:38:14.191 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cherrypy3. 2020-05-18T21:38:14.211 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../25-python3-cherrypy3_8.9.1-6_all.deb ... 2020-05-18T21:38:14.221 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-werkzeug. 2020-05-18T21:38:14.237 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../41-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2020-05-18T21:38:14.249 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2020-05-18T21:38:14.297 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2020-05-18T21:38:14.603 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-logutils. 2020-05-18T21:38:14.620 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../26-python3-logutils_0.3.3-7_all.deb ... 2020-05-18T21:38:14.661 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-logutils (0.3.3-7) ... 2020-05-18T21:38:14.682 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr. 2020-05-18T21:38:14.700 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../42-ceph-mgr_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:14.740 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:14.964 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-markupsafe. 2020-05-18T21:38:14.981 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../27-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2020-05-18T21:38:15.023 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2020-05-18T21:38:15.252 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mon. 2020-05-18T21:38:15.268 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../43-ceph-mon_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:15.310 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mon (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:15.326 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-mako. 2020-05-18T21:38:15.343 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../28-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2020-05-18T21:38:15.384 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2020-05-18T21:38:15.779 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-simplegeneric. 2020-05-18T21:38:15.796 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../29-python3-simplegeneric_0.8.1-3_all.deb ... 2020-05-18T21:38:15.837 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2020-05-18T21:38:16.182 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-singledispatch. 2020-05-18T21:38:16.201 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../30-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2020-05-18T21:38:16.240 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2020-05-18T21:38:16.330 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-osd. 2020-05-18T21:38:16.350 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../44-ceph-osd_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:16.388 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-osd (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:16.560 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-webob. 2020-05-18T21:38:16.580 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../31-python3-webob_1%3a1.8.5-2_all.deb ... 2020-05-18T21:38:16.618 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2020-05-18T21:38:16.922 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-soupsieve. 2020-05-18T21:38:16.941 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../32-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2020-05-18T21:38:16.982 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2020-05-18T21:38:17.258 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-bs4. 2020-05-18T21:38:17.276 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../33-python3-bs4_4.8.2-1_all.deb ... 2020-05-18T21:38:17.316 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2020-05-18T21:38:17.678 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-waitress. 2020-05-18T21:38:17.699 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../34-python3-waitress_1.4.1-1_all.deb ... 2020-05-18T21:38:17.739 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-waitress (1.4.1-1) ... 2020-05-18T21:38:18.064 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-tempita. 2020-05-18T21:38:18.082 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../35-python3-tempita_0.5.2-6_all.deb ... 2020-05-18T21:38:18.122 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-tempita (0.5.2-6) ... 2020-05-18T21:38:18.426 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-paste. 2020-05-18T21:38:18.444 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../36-python3-paste_3.4.0+dfsg1-1_all.deb ... 2020-05-18T21:38:18.484 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2020-05-18T21:38:18.744 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph. 2020-05-18T21:38:18.761 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../45-ceph_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:18.801 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:18.813 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-05-18T21:38:18.830 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../37-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2020-05-18T21:38:18.871 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2020-05-18T21:38:19.104 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-fuse. 2020-05-18T21:38:19.120 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../46-ceph-fuse_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:19.162 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-fuse (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:19.191 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pastedeploy. 2020-05-18T21:38:19.211 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../38-python3-pastedeploy_2.1.0-1_all.deb ... 2020-05-18T21:38:19.248 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2020-05-18T21:38:19.583 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mds. 2020-05-18T21:38:19.585 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-webtest. 2020-05-18T21:38:19.600 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../47-ceph-mds_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:19.604 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../39-python3-webtest_2.0.32-2_all.deb ... 2020-05-18T21:38:19.640 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mds (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:19.643 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-webtest (2.0.32-2) ... 2020-05-18T21:38:19.947 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pecan. 2020-05-18T21:38:19.963 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../40-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2020-05-18T21:38:20.005 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2020-05-18T21:38:20.127 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephadm. 2020-05-18T21:38:20.144 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../48-cephadm_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:20.185 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephadm (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:20.335 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-werkzeug. 2020-05-18T21:38:20.355 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../41-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2020-05-18T21:38:20.393 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2020-05-18T21:38:20.513 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-05-18T21:38:20.532 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../49-ceph-mgr-cephadm_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:20.571 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-cephadm (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:20.822 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr. 2020-05-18T21:38:20.843 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../42-ceph-mgr_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:20.880 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:20.899 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-repoze.lru. 2020-05-18T21:38:20.919 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../50-python3-repoze.lru_0.7-2_all.deb ... 2020-05-18T21:38:20.957 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2020-05-18T21:38:21.160 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libjs-underscore. 2020-05-18T21:38:21.179 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../51-libjs-underscore_1.9.1~dfsg-1_all.deb ... 2020-05-18T21:38:21.243 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1) ... 2020-05-18T21:38:21.358 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mon. 2020-05-18T21:38:21.378 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../43-ceph-mon_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:21.417 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mon (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:21.496 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libjs-sphinxdoc. 2020-05-18T21:38:21.512 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../52-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2020-05-18T21:38:21.554 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2020-05-18T21:38:21.874 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-routes. 2020-05-18T21:38:21.896 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../53-python3-routes_2.4.1-2_all.deb ... 2020-05-18T21:38:21.932 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-routes (2.4.1-2) ... 2020-05-18T21:38:22.286 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-05-18T21:38:22.304 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../54-ceph-mgr-dashboard_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:22.350 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-dashboard (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:22.429 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-osd. 2020-05-18T21:38:22.445 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../44-ceph-osd_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:22.487 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-osd (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:23.318 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2020-05-18T21:38:23.336 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-cloud_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:23.384 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-diskprediction-cloud (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:23.704 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-decorator. 2020-05-18T21:38:23.723 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../56-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2020-05-18T21:38:23.762 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2020-05-18T21:38:24.090 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2020-05-18T21:38:24.108 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../57-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2020-05-18T21:38:24.148 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2020-05-18T21:38:24.451 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-scipy. 2020-05-18T21:38:24.471 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../58-python3-scipy_1.3.3-3build1_amd64.deb ... 2020-05-18T21:38:24.509 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2020-05-18T21:38:24.934 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph. 2020-05-18T21:38:24.954 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../45-ceph_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:24.992 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:25.262 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-fuse. 2020-05-18T21:38:25.280 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../46-ceph-fuse_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:25.320 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-fuse (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:25.741 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mds. 2020-05-18T21:38:25.760 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../47-ceph-mds_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:25.799 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mds (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:25.810 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-05-18T21:38:25.828 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../59-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2020-05-18T21:38:25.868 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2020-05-18T21:38:26.319 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package cephadm. 2020-05-18T21:38:26.336 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../48-cephadm_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:26.385 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking cephadm (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:26.497 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-joblib. 2020-05-18T21:38:26.518 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../60-python3-joblib_0.14.0-3_all.deb ... 2020-05-18T21:38:26.555 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-joblib (0.14.0-3) ... 2020-05-18T21:38:26.689 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-05-18T21:38:26.705 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../49-ceph-mgr-cephadm_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:26.772 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-cephadm (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:26.925 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-sklearn. 2020-05-18T21:38:26.946 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../61-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2020-05-18T21:38:26.983 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2020-05-18T21:38:27.125 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-repoze.lru. 2020-05-18T21:38:27.144 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../50-python3-repoze.lru_0.7-2_all.deb ... 2020-05-18T21:38:27.183 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2020-05-18T21:38:27.429 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libjs-underscore. 2020-05-18T21:38:27.445 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../51-libjs-underscore_1.9.1~dfsg-1_all.deb ... 2020-05-18T21:38:27.488 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1) ... 2020-05-18T21:38:27.717 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-05-18T21:38:27.734 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../62-ceph-mgr-diskprediction-local_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:27.775 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:27.806 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libjs-sphinxdoc. 2020-05-18T21:38:27.826 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../52-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2020-05-18T21:38:27.864 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2020-05-18T21:38:28.184 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-routes. 2020-05-18T21:38:28.201 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../53-python3-routes_2.4.1-2_all.deb ... 2020-05-18T21:38:28.242 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-routes (2.4.1-2) ... 2020-05-18T21:38:28.563 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-05-18T21:38:28.583 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../54-ceph-mgr-dashboard_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:28.621 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-dashboard (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:28.895 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cachetools. 2020-05-18T21:38:28.912 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../63-python3-cachetools_4.0.0-1_all.deb ... 2020-05-18T21:38:28.952 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2020-05-18T21:38:29.222 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyasn1. 2020-05-18T21:38:29.240 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../64-python3-pyasn1_0.4.2-3build1_all.deb ... 2020-05-18T21:38:29.289 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2020-05-18T21:38:29.507 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2020-05-18T21:38:29.528 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-cloud_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:29.560 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-05-18T21:38:29.565 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-diskprediction-cloud (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:29.580 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../65-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2020-05-18T21:38:29.618 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2020-05-18T21:38:29.901 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-decorator. 2020-05-18T21:38:29.918 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../56-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2020-05-18T21:38:29.954 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rsa. 2020-05-18T21:38:29.959 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2020-05-18T21:38:29.971 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../66-python3-rsa_4.0-3_all.deb ... 2020-05-18T21:38:30.012 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rsa (4.0-3) ... 2020-05-18T21:38:30.271 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2020-05-18T21:38:30.289 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../57-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2020-05-18T21:38:30.332 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-google-auth. 2020-05-18T21:38:30.337 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2020-05-18T21:38:30.351 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../67-python3-google-auth_1.5.1-2_all.deb ... 2020-05-18T21:38:30.390 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2020-05-18T21:38:30.624 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-scipy. 2020-05-18T21:38:30.641 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../58-python3-scipy_1.3.3-3build1_amd64.deb ... 2020-05-18T21:38:30.682 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2020-05-18T21:38:30.727 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-requests-oauthlib. 2020-05-18T21:38:30.744 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../68-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2020-05-18T21:38:30.802 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2020-05-18T21:38:31.114 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-websocket. 2020-05-18T21:38:31.130 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../69-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2020-05-18T21:38:31.171 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2020-05-18T21:38:31.483 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-kubernetes. 2020-05-18T21:38:31.500 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../70-python3-kubernetes_7.0.0~a1-2_all.deb ... 2020-05-18T21:38:31.672 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2020-05-18T21:38:32.425 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-05-18T21:38:32.442 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../59-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2020-05-18T21:38:32.471 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-05-18T21:38:32.474 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2020-05-18T21:38:32.491 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../71-ceph-mgr-k8sevents_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:32.529 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:32.873 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-json-pointer. 2020-05-18T21:38:32.892 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../72-python3-json-pointer_2.0-0ubuntu1_all.deb ... 2020-05-18T21:38:32.931 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-json-pointer (2.0-0ubuntu1) ... 2020-05-18T21:38:33.095 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-joblib. 2020-05-18T21:38:33.113 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../60-python3-joblib_0.14.0-3_all.deb ... 2020-05-18T21:38:33.153 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-joblib (0.14.0-3) ... 2020-05-18T21:38:33.276 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jsonpatch. 2020-05-18T21:38:33.295 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../73-python3-jsonpatch_1.23-3_all.deb ... 2020-05-18T21:38:33.343 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jsonpatch (1.23-3) ... 2020-05-18T21:38:33.515 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-sklearn. 2020-05-18T21:38:33.533 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../61-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2020-05-18T21:38:33.573 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2020-05-18T21:38:33.654 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-05-18T21:38:33.671 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../74-ceph-mgr-rook_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:33.712 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-rook (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:34.049 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libonig5:amd64. 2020-05-18T21:38:34.067 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../75-libonig5_6.9.4-1_amd64.deb ... 2020-05-18T21:38:34.107 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2020-05-18T21:38:34.332 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-05-18T21:38:34.351 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../62-ceph-mgr-diskprediction-local_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:34.390 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:34.485 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libjq1:amd64. 2020-05-18T21:38:34.501 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../76-libjq1_1.6-1_amd64.deb ... 2020-05-18T21:38:34.543 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libjq1:amd64 (1.6-1) ... 2020-05-18T21:38:34.829 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package jq. 2020-05-18T21:38:34.849 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../77-jq_1.6-1_amd64.deb ... 2020-05-18T21:38:34.887 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking jq (1.6-1) ... 2020-05-18T21:38:35.198 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package socat. 2020-05-18T21:38:35.216 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../78-socat_1.7.3.3-2_amd64.deb ... 2020-05-18T21:38:35.256 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking socat (1.7.3.3-2) ... 2020-05-18T21:38:35.502 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cachetools. 2020-05-18T21:38:35.523 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../63-python3-cachetools_4.0.0-1_all.deb ... 2020-05-18T21:38:35.551 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package xmlstarlet. 2020-05-18T21:38:35.560 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2020-05-18T21:38:35.574 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../79-xmlstarlet_1.6.1-2build1_amd64.deb ... 2020-05-18T21:38:35.609 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2020-05-18T21:38:35.889 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyasn1. 2020-05-18T21:38:35.906 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../64-python3-pyasn1_0.4.2-3build1_all.deb ... 2020-05-18T21:38:35.938 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-test. 2020-05-18T21:38:35.947 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2020-05-18T21:38:35.959 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../80-ceph-test_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:35.996 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-test (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:36.310 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-05-18T21:38:36.326 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../65-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2020-05-18T21:38:36.367 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2020-05-18T21:38:36.663 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rsa. 2020-05-18T21:38:36.683 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../66-python3-rsa_4.0-3_all.deb ... 2020-05-18T21:38:36.720 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rsa (4.0-3) ... 2020-05-18T21:38:37.049 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-google-auth. 2020-05-18T21:38:37.068 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../67-python3-google-auth_1.5.1-2_all.deb ... 2020-05-18T21:38:37.107 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2020-05-18T21:38:37.453 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-requests-oauthlib. 2020-05-18T21:38:37.472 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../68-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2020-05-18T21:38:37.519 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2020-05-18T21:38:37.839 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-websocket. 2020-05-18T21:38:37.857 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../69-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2020-05-18T21:38:37.897 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2020-05-18T21:38:38.217 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-kubernetes. 2020-05-18T21:38:38.236 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../70-python3-kubernetes_7.0.0~a1-2_all.deb ... 2020-05-18T21:38:38.275 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2020-05-18T21:38:38.789 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-05-18T21:38:38.806 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../71-ceph-mgr-k8sevents_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:38.847 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:39.200 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-json-pointer. 2020-05-18T21:38:39.217 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../72-python3-json-pointer_2.0-0ubuntu1_all.deb ... 2020-05-18T21:38:39.258 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-json-pointer (2.0-0ubuntu1) ... 2020-05-18T21:38:39.552 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-colorama. 2020-05-18T21:38:39.571 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../81-python3-colorama_0.4.3-1build1_all.deb ... 2020-05-18T21:38:39.586 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jsonpatch. 2020-05-18T21:38:39.604 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../73-python3-jsonpatch_1.23-3_all.deb ... 2020-05-18T21:38:39.609 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2020-05-18T21:38:39.644 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jsonpatch (1.23-3) ... 2020-05-18T21:38:39.871 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephfs-shell. 2020-05-18T21:38:39.888 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../82-cephfs-shell_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:39.929 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephfs-shell (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:39.989 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-05-18T21:38:40.008 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../74-ceph-mgr-rook_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:40.047 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-rook (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:40.190 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libcephfs-dev. 2020-05-18T21:38:40.208 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../83-libcephfs-dev_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:40.250 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libcephfs-dev (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:40.384 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libonig5:amd64. 2020-05-18T21:38:40.403 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../75-libonig5_6.9.4-1_amd64.deb ... 2020-05-18T21:38:40.442 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2020-05-18T21:38:40.593 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-atomicwrites. 2020-05-18T21:38:40.610 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../84-python3-atomicwrites_1.1.5-2build1_all.deb ... 2020-05-18T21:38:40.651 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2020-05-18T21:38:40.795 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libjq1:amd64. 2020-05-18T21:38:40.813 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../76-libjq1_1.6-1_amd64.deb ... 2020-05-18T21:38:40.853 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libjq1:amd64 (1.6-1) ... 2020-05-18T21:38:40.970 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-attr. 2020-05-18T21:38:40.991 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../85-python3-attr_19.3.0-2_all.deb ... 2020-05-18T21:38:41.028 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-attr (19.3.0-2) ... 2020-05-18T21:38:41.165 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package jq. 2020-05-18T21:38:41.184 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../77-jq_1.6-1_amd64.deb ... 2020-05-18T21:38:41.222 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking jq (1.6-1) ... 2020-05-18T21:38:41.376 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-webencodings. 2020-05-18T21:38:41.395 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../86-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2020-05-18T21:38:41.433 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2020-05-18T21:38:41.484 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package socat. 2020-05-18T21:38:41.501 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../78-socat_1.7.3.3-2_amd64.deb ... 2020-05-18T21:38:41.542 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking socat (1.7.3.3-2) ... 2020-05-18T21:38:41.761 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-html5lib. 2020-05-18T21:38:41.779 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../87-python3-html5lib_1.0.1-2_all.deb ... 2020-05-18T21:38:41.819 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2020-05-18T21:38:41.862 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package xmlstarlet. 2020-05-18T21:38:41.879 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../79-xmlstarlet_1.6.1-2build1_amd64.deb ... 2020-05-18T21:38:41.920 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2020-05-18T21:38:42.231 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-05-18T21:38:42.248 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../88-python3-lxml_4.5.0-1_amd64.deb ... 2020-05-18T21:38:42.297 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-lxml:amd64 (4.5.0-1) ... 2020-05-18T21:38:42.300 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-test. 2020-05-18T21:38:42.318 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../80-ceph-test_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:42.358 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-test (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:42.792 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-packaging. 2020-05-18T21:38:42.814 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../89-python3-packaging_20.3-1_all.deb ... 2020-05-18T21:38:42.850 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-packaging (20.3-1) ... 2020-05-18T21:38:43.212 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pastescript. 2020-05-18T21:38:43.231 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../90-python3-pastescript_2.0.2-4_all.deb ... 2020-05-18T21:38:43.270 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2020-05-18T21:38:43.648 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pluggy. 2020-05-18T21:38:43.668 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../91-python3-pluggy_0.13.0-2_all.deb ... 2020-05-18T21:38:43.714 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2020-05-18T21:38:44.051 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-psutil. 2020-05-18T21:38:44.069 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../92-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2020-05-18T21:38:44.109 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2020-05-18T21:38:44.429 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-py. 2020-05-18T21:38:44.450 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../93-python3-py_1.8.1-1_all.deb ... 2020-05-18T21:38:44.487 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-py (1.8.1-1) ... 2020-05-18T21:38:44.832 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyinotify. 2020-05-18T21:38:44.849 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../94-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2020-05-18T21:38:44.890 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2020-05-18T21:38:45.202 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pytest. 2020-05-18T21:38:45.222 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../95-python3-pytest_4.6.9-1_all.deb ... 2020-05-18T21:38:45.259 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pytest (4.6.9-1) ... 2020-05-18T21:38:45.722 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package radosgw. 2020-05-18T21:38:45.742 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../96-radosgw_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:45.780 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking radosgw (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:46.021 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-colorama. 2020-05-18T21:38:46.038 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../81-python3-colorama_0.4.3-1build1_all.deb ... 2020-05-18T21:38:46.079 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2020-05-18T21:38:46.307 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package cephfs-shell. 2020-05-18T21:38:46.327 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../82-cephfs-shell_16.0.0-1640-g543315c9344-1focal_all.deb ... 2020-05-18T21:38:46.357 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking cephfs-shell (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:46.610 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libcephfs-dev. 2020-05-18T21:38:46.628 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../83-libcephfs-dev_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:46.668 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libcephfs-dev (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:46.850 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package rbd-fuse. 2020-05-18T21:38:46.870 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../97-rbd-fuse_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:46.908 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking rbd-fuse (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:46.971 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-atomicwrites. 2020-05-18T21:38:46.991 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../84-python3-atomicwrites_1.1.5-2build1_all.deb ... 2020-05-18T21:38:47.029 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2020-05-18T21:38:47.227 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-attr (19.3.0-2) ... 2020-05-18T21:38:47.316 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-attr. 2020-05-18T21:38:47.334 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../85-python3-attr_19.3.0-2_all.deb ... 2020-05-18T21:38:47.365 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-attr (19.3.0-2) ... 2020-05-18T21:38:47.587 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2020-05-18T21:38:47.686 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-webencodings. 2020-05-18T21:38:47.703 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../86-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2020-05-18T21:38:47.730 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2020-05-18T21:38:47.749 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2020-05-18T21:38:47.856 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephadm (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:48.038 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-html5lib. 2020-05-18T21:38:48.056 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../87-python3-html5lib_1.0.1-2_all.deb ... 2020-05-18T21:38:48.096 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2020-05-18T21:38:48.458 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-05-18T21:38:48.475 INFO:teuthology.orchestra.run.smithi196.stdout:Adding system user cephadm....done 2020-05-18T21:38:48.478 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../88-python3-lxml_4.5.0-1_amd64.deb ... 2020-05-18T21:38:48.516 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-lxml:amd64 (4.5.0-1) ... 2020-05-18T21:38:48.637 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-waitress (1.4.1-1) ... 2020-05-18T21:38:48.986 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-packaging. 2020-05-18T21:38:49.007 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../89-python3-packaging_20.3-1_all.deb ... 2020-05-18T21:38:49.038 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-repoze.lru (0.7-2) ... 2020-05-18T21:38:49.044 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-packaging (20.3-1) ... 2020-05-18T21:38:49.339 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liboath0 (2.6.1-1.3) ... 2020-05-18T21:38:49.373 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pastescript. 2020-05-18T21:38:49.389 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../90-python3-pastescript_2.0.2-4_all.deb ... 2020-05-18T21:38:49.430 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2020-05-18T21:38:49.465 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-py (1.8.1-1) ... 2020-05-18T21:38:49.734 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pluggy. 2020-05-18T21:38:49.755 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../91-python3-pluggy_0.13.0-2_all.deb ... 2020-05-18T21:38:49.784 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2020-05-18T21:38:49.809 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-joblib (0.14.0-3) ... 2020-05-18T21:38:50.054 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-psutil. 2020-05-18T21:38:50.075 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../92-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2020-05-18T21:38:50.112 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2020-05-18T21:38:50.235 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cachetools (4.0.0-1) ... 2020-05-18T21:38:50.457 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-py. 2020-05-18T21:38:50.474 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../93-python3-py_1.8.1-1_all.deb ... 2020-05-18T21:38:50.515 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-py (1.8.1-1) ... 2020-05-18T21:38:50.536 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2020-05-18T21:38:50.802 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyinotify. 2020-05-18T21:38:50.819 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../94-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2020-05-18T21:38:50.820 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2020-05-18T21:38:50.851 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2020-05-18T21:38:51.138 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2020-05-18T21:38:51.155 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pytest. 2020-05-18T21:38:51.175 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../95-python3-pytest_4.6.9-1_all.deb ... 2020-05-18T21:38:51.213 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pytest (4.6.9-1) ... 2020-05-18T21:38:51.431 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2020-05-18T21:38:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package radosgw. 2020-05-18T21:38:51.661 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../96-radosgw_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:51.700 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking radosgw (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:51.741 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-ceph-argparse (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:52.050 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2020-05-18T21:38:52.193 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2020-05-18T21:38:52.552 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2020-05-18T21:38:52.845 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2020-05-18T21:38:52.862 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package rbd-fuse. 2020-05-18T21:38:52.881 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../97-rbd-fuse_16.0.0-1640-g543315c9344-1focal_amd64.deb ... 2020-05-18T21:38:52.920 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking rbd-fuse (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:53.129 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2020-05-18T21:38:53.209 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-attr (19.3.0-2) ... 2020-05-18T21:38:53.514 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2020-05-18T21:38:53.542 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2020-05-18T21:38:53.668 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2020-05-18T21:38:53.811 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up cephadm (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:53.815 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2020-05-18T21:38:54.099 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-tempita (0.5.2-6) ... 2020-05-18T21:38:54.364 INFO:teuthology.orchestra.run.smithi080.stdout:Adding system user cephadm....done 2020-05-18T21:38:54.384 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-packaging (20.3-1) ... 2020-05-18T21:38:54.559 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-waitress (1.4.1-1) ... 2020-05-18T21:38:54.685 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2020-05-18T21:38:54.811 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2020-05-18T21:38:54.894 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-repoze.lru (0.7-2) ... 2020-05-18T21:38:55.112 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2020-05-18T21:38:55.203 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liboath0 (2.6.1-1.3) ... 2020-05-18T21:38:55.330 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-py (1.8.1-1) ... 2020-05-18T21:38:55.514 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2020-05-18T21:38:55.665 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-joblib (0.14.0-3) ... 2020-05-18T21:38:55.823 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2020-05-18T21:38:56.075 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cachetools (4.0.0-1) ... 2020-05-18T21:38:56.183 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up socat (1.7.3.3-2) ... 2020-05-18T21:38:56.283 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-ceph-common (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:56.368 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2020-05-18T21:38:56.585 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-html5lib (1.0.1-2) ... 2020-05-18T21:38:56.661 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2020-05-18T21:38:56.953 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2020-05-18T21:38:56.962 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2020-05-18T21:38:57.079 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2020-05-18T21:38:57.205 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pluggy (0.13.0-2) ... 2020-05-18T21:38:57.238 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2020-05-18T21:38:57.498 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-json-pointer (2.0-0ubuntu1) ... 2020-05-18T21:38:57.572 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-ceph-argparse (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:57.790 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-lxml:amd64 (4.5.0-1) ... 2020-05-18T21:38:57.882 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2020-05-18T21:38:58.008 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2020-05-18T21:38:58.092 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2020-05-18T21:38:58.309 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2020-05-18T21:38:58.402 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2020-05-18T21:38:58.619 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2020-05-18T21:38:58.694 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-logutils (0.3.3-7) ... 2020-05-18T21:38:58.886 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2020-05-18T21:38:58.995 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-modules-core (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:38:59.121 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2020-05-18T21:38:59.246 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2020-05-18T21:38:59.263 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2020-05-18T21:38:59.531 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2020-05-18T21:38:59.557 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-prettytable (0.7.2-5) ... 2020-05-18T21:38:59.840 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-tempita (0.5.2-6) ... 2020-05-18T21:38:59.858 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2020-05-18T21:39:00.124 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-packaging (20.3-1) ... 2020-05-18T21:39:00.167 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2020-05-18T21:39:00.418 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2020-05-18T21:39:00.538 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-05-18T21:39:00.543 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2020-05-18T21:39:00.803 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2020-05-18T21:39:00.820 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2020-05-18T21:39:00.929 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2020-05-18T21:39:01.205 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2020-05-18T21:39:01.222 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2020-05-18T21:39:01.514 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2020-05-18T21:39:01.565 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2020-05-18T21:39:01.891 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up socat (1.7.3.3-2) ... 2020-05-18T21:39:01.925 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libjs-underscore (1.9.1~dfsg-1) ... 2020-05-18T21:39:02.017 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-ceph-common (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:02.084 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2020-05-18T21:39:02.210 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-webob (1:1.8.5-2) ... 2020-05-18T21:39:02.326 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-html5lib (1.0.1-2) ... 2020-05-18T21:39:02.678 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2020-05-18T21:39:02.794 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librados2 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:02.804 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2020-05-18T21:39:02.922 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pluggy (0.13.0-2) ... 2020-05-18T21:39:03.029 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2020-05-18T21:39:03.623 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-json-pointer (2.0-0ubuntu1) ... 2020-05-18T21:39:03.907 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-lxml:amd64 (4.5.0-1) ... 2020-05-18T21:39:04.242 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2020-05-18T21:39:04.510 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librgw2 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:04.569 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2020-05-18T21:39:04.669 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libjq1:amd64 (1.6-1) ... 2020-05-18T21:39:04.795 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pytest (4.6.9-1) ... 2020-05-18T21:39:04.837 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-logutils (0.3.3-7) ... 2020-05-18T21:39:05.104 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-modules-core (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:05.171 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jsonpatch (1.23-3) ... 2020-05-18T21:39:05.230 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2020-05-18T21:39:05.261 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/jsonpatch-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-05-18T21:39:05.356 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2020-05-18T21:39:05.523 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2020-05-18T21:39:05.666 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-prettytable (0.7.2-5) ... 2020-05-18T21:39:05.816 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-bs4 (4.8.2-1) ... 2020-05-18T21:39:05.967 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2020-05-18T21:39:06.192 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libcephfs2 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:06.285 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2020-05-18T21:39:06.318 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libradosstriper1 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:06.469 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2020-05-18T21:39:06.643 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-05-18T21:39:06.804 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2020-05-18T21:39:06.905 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2020-05-18T21:39:07.031 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2020-05-18T21:39:07.332 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2020-05-18T21:39:07.650 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2020-05-18T21:39:07.817 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librbd1 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:07.952 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2020-05-18T21:39:08.018 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libjs-underscore (1.9.1~dfsg-1) ... 2020-05-18T21:39:08.103 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up jq (1.6-1) ... 2020-05-18T21:39:08.228 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rsa (4.0-3) ... 2020-05-18T21:39:08.270 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2020-05-18T21:39:08.379 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-webob (1:1.8.5-2) ... 2020-05-18T21:39:08.538 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-webtest (2.0.32-2) ... 2020-05-18T21:39:08.697 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librados2 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:08.823 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2020-05-18T21:39:08.856 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-fuse (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:09.115 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-05-18T21:39:09.115 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-05-18T21:39:09.566 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libcephfs-dev (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:09.692 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pastescript (2.0.2-4) ... 2020-05-18T21:39:10.018 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2020-05-18T21:39:10.321 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librgw2 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:10.370 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rados (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:10.446 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libjq1:amd64 (1.6-1) ... 2020-05-18T21:39:10.496 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rbd (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:10.572 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pytest (4.6.9-1) ... 2020-05-18T21:39:10.622 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up rbd-fuse (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:10.782 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-routes (2.4.1-2) ... 2020-05-18T21:39:10.949 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jsonpatch (1.23-3) ... 2020-05-18T21:39:11.039 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/jsonpatch-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-05-18T21:39:11.091 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-google-auth (1.5.1-2) ... 2020-05-18T21:39:11.267 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2020-05-18T21:39:11.384 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rgw (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:11.518 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cephfs (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:11.544 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-bs4 (4.8.2-1) ... 2020-05-18T21:39:11.771 INFO:teuthology.orchestra.run.smithi196.stdout:/usr/lib/python3/dist-packages/ceph_volume_client.py:358: SyntaxWarning: "is not" with a literal. Did you mean "!="? 2020-05-18T21:39:11.772 INFO:teuthology.orchestra.run.smithi196.stdout: group_id = group_id if group_id is not 'None' else None 2020-05-18T21:39:11.772 INFO:teuthology.orchestra.run.smithi196.stdout:/usr/lib/python3/dist-packages/ceph_volume_client.py:381: SyntaxWarning: "is" with a literal. Did you mean "=="? 2020-05-18T21:39:11.772 INFO:teuthology.orchestra.run.smithi196.stdout: readonly = True if access_level is 'r' else False 2020-05-18T21:39:11.772 INFO:teuthology.orchestra.run.smithi196.stdout:/usr/lib/python3/dist-packages/ceph_volume_client.py:1102: SyntaxWarning: "is" with a literal. Did you mean "=="? 2020-05-18T21:39:11.773 INFO:teuthology.orchestra.run.smithi196.stdout: unwanted_access_level = 'r' if want_access_level is 'rw' else 'rw' 2020-05-18T21:39:11.836 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephfs-shell (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:11.895 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libcephfs2 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:11.962 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2020-05-18T21:39:12.046 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libradosstriper1 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:12.172 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2020-05-18T21:39:12.498 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2020-05-18T21:39:12.882 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-common (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:13.545 INFO:teuthology.orchestra.run.smithi196.stdout:Adding group ceph....done 2020-05-18T21:39:13.554 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librbd1 (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:13.688 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2020-05-18T21:39:13.814 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up jq (1.6-1) ... 2020-05-18T21:39:13.921 INFO:teuthology.orchestra.run.smithi196.stdout:Adding system user ceph....done 2020-05-18T21:39:13.940 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rsa (4.0-3) ... 2020-05-18T21:39:14.072 INFO:teuthology.orchestra.run.smithi196.stdout:Setting system user ceph properties....done 2020-05-18T21:39:14.242 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-05-18T21:39:14.258 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-webtest (2.0.32-2) ... 2020-05-18T21:39:14.551 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-05-18T21:39:14.559 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-fuse (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:14.769 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-05-18T21:39:14.769 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-05-18T21:39:15.027 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-test (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:15.169 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up radosgw (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:15.236 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libcephfs-dev (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:15.362 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pastescript (2.0.2-4) ... 2020-05-18T21:39:15.440 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-05-18T21:39:15.440 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-05-18T21:39:15.664 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2020-05-18T21:39:16.024 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rados (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:16.154 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-base (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:16.175 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rbd (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:16.302 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up rbd-fuse (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:16.428 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-routes (2.4.1-2) ... 2020-05-18T21:39:16.472 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-05-18T21:39:16.721 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-google-auth (1.5.1-2) ... 2020-05-18T21:39:16.931 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mds (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:17.030 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rgw (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:17.132 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-05-18T21:39:17.132 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-05-18T21:39:17.190 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cephfs (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:17.427 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/lib/python3/dist-packages/ceph_volume_client.py:358: SyntaxWarning: "is not" with a literal. Did you mean "!="? 2020-05-18T21:39:17.427 INFO:teuthology.orchestra.run.smithi080.stdout: group_id = group_id if group_id is not 'None' else None 2020-05-18T21:39:17.427 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/lib/python3/dist-packages/ceph_volume_client.py:381: SyntaxWarning: "is" with a literal. Did you mean "=="? 2020-05-18T21:39:17.427 INFO:teuthology.orchestra.run.smithi080.stdout: readonly = True if access_level is 'r' else False 2020-05-18T21:39:17.428 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/lib/python3/dist-packages/ceph_volume_client.py:1102: SyntaxWarning: "is" with a literal. Did you mean "=="? 2020-05-18T21:39:17.428 INFO:teuthology.orchestra.run.smithi080.stdout: unwanted_access_level = 'r' if want_access_level is 'rw' else 'rw' 2020-05-18T21:39:17.499 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up cephfs-shell (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:17.624 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:17.625 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2020-05-18T21:39:17.830 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-05-18T21:39:17.831 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-05-18T21:39:18.284 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-diskprediction-cloud (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:18.393 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-rook (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:18.547 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-common (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:18.552 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-osd (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:19.130 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-05-18T21:39:19.130 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-05-18T21:39:19.211 INFO:teuthology.orchestra.run.smithi080.stdout:Adding group ceph....done 2020-05-18T21:39:19.579 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-k8sevents (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:19.595 INFO:teuthology.orchestra.run.smithi080.stdout:Adding system user ceph....done 2020-05-18T21:39:19.689 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:19.764 INFO:teuthology.orchestra.run.smithi080.stdout:Setting system user ceph properties....done 2020-05-18T21:39:19.831 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mon (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:19.933 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-05-18T21:39:20.059 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-05-18T21:39:20.060 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-05-18T21:39:20.245 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-05-18T21:39:20.516 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-cephadm (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:20.658 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:20.701 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-test (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:20.784 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-dashboard (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:20.802 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up radosgw (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:20.928 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for systemd (245.4-4ubuntu3) ... 2020-05-18T21:39:21.064 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-05-18T21:39:21.064 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-05-18T21:39:21.237 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for man-db (2.9.1-1) ... 2020-05-18T21:39:21.754 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-base (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:22.083 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-05-18T21:39:22.531 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mds (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:22.623 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for libc-bin (2.31-0ubuntu9) ... 2020-05-18T21:39:22.769 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-05-18T21:39:22.770 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-05-18T21:39:23.048 INFO:teuthology.orchestra.run.smithi196.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-05-18T21:39:23.051 INFO:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-05-18T21:39:23.121 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2020-05-18T21:39:23.249 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:23.255 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2020-05-18T21:39:23.256 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2020-05-18T21:39:23.383 INFO:teuthology.orchestra.run.smithi196.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2020-05-18T21:39:23.384 INFO:teuthology.orchestra.run.smithi196.stdout:libtool-bin is already the newest version (2.4.6-14). 2020-05-18T21:39:23.384 INFO:teuthology.orchestra.run.smithi196.stdout:network-manager is already the newest version (1.22.10-1ubuntu1). 2020-05-18T21:39:23.384 INFO:teuthology.orchestra.run.smithi196.stdout:network-manager set to manually installed. 2020-05-18T21:39:23.384 INFO:teuthology.orchestra.run.smithi196.stdout:util-linux is already the newest version (2.34-0.1ubuntu9). 2020-05-18T21:39:23.384 INFO:teuthology.orchestra.run.smithi196.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9). 2020-05-18T21:39:23.385 INFO:teuthology.orchestra.run.smithi196.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2020-05-18T21:39:23.385 INFO:teuthology.orchestra.run.smithi196.stdout:The following package was automatically installed and is no longer required: 2020-05-18T21:39:23.386 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.71.0 2020-05-18T21:39:23.386 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove it. 2020-05-18T21:39:23.386 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2020-05-18T21:39:23.386 INFO:teuthology.orchestra.run.smithi196.stdout: libfl-dev libfl2 liblzo2-2 2020-05-18T21:39:23.387 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2020-05-18T21:39:23.387 INFO:teuthology.orchestra.run.smithi196.stdout: bison-doc flex-doc indent-doc libssl-doc 2020-05-18T21:39:23.406 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2020-05-18T21:39:23.407 INFO:teuthology.orchestra.run.smithi196.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2020-05-18T21:39:23.407 INFO:teuthology.orchestra.run.smithi196.stdout: libssl-dev 2020-05-18T21:39:23.477 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-05-18T21:39:23.477 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-05-18T21:39:23.588 INFO:teuthology.orchestra.run.smithi196.stdout:0 upgraded, 10 newly installed, 0 to remove and 45 not upgraded. 2020-05-18T21:39:23.588 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 2,912 kB of archives. 2020-05-18T21:39:23.588 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 12.4 MB of additional disk space will be used. 2020-05-18T21:39:23.589 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2020-05-18T21:39:23.942 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-diskprediction-cloud (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:24.055 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2020-05-18T21:39:24.085 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-rook (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:24.189 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2020-05-18T21:39:24.196 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2020-05-18T21:39:24.213 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2020-05-18T21:39:24.221 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2020-05-18T21:39:24.222 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2020-05-18T21:39:24.227 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2020-05-18T21:39:24.228 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2020-05-18T21:39:24.228 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2 [1,582 kB] 2020-05-18T21:39:24.251 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-osd (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:24.709 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 2,912 kB in 1s (3,184 kB/s) 2020-05-18T21:39:24.818 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-05-18T21:39:24.818 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-05-18T21:39:24.854 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package flex. 2020-05-18T21:39:24.919 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 156462 files and directories currently installed.) 2020-05-18T21:39:24.921 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2020-05-18T21:39:24.960 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking flex (2.6.4-6.2) ... 2020-05-18T21:39:25.305 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-k8sevents (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:25.448 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:25.506 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package bison. 2020-05-18T21:39:25.523 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2020-05-18T21:39:25.569 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2020-05-18T21:39:25.607 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mon (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:25.829 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-05-18T21:39:25.829 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-05-18T21:39:25.993 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblzo2-2:amd64. 2020-05-18T21:39:26.011 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2020-05-18T21:39:26.093 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2020-05-18T21:39:26.323 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-cephadm (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:26.379 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package dump. 2020-05-18T21:39:26.396 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2020-05-18T21:39:26.429 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking dump (0.4b46-6) ... 2020-05-18T21:39:26.468 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:26.594 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-dashboard (16.0.0-1640-g543315c9344-1focal) ... 2020-05-18T21:39:26.716 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package indent. 2020-05-18T21:39:26.733 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2020-05-18T21:39:26.739 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for systemd (245.4-4ubuntu3) ... 2020-05-18T21:39:26.774 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking indent (2.2.12-1) ... 2020-05-18T21:39:27.043 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-05-18T21:39:27.056 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for man-db (2.9.1-1) ... 2020-05-18T21:39:27.063 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2020-05-18T21:39:27.202 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2020-05-18T21:39:27.463 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-05-18T21:39:27.482 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2020-05-18T21:39:27.521 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2020-05-18T21:39:27.824 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libfl2:amd64. 2020-05-18T21:39:27.843 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2020-05-18T21:39:27.882 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2020-05-18T21:39:28.160 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-05-18T21:39:28.177 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2020-05-18T21:39:28.218 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2020-05-18T21:39:28.466 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for libc-bin (2.31-0ubuntu9) ... 2020-05-18T21:39:28.572 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-05-18T21:39:28.590 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2_amd64.deb ... 2020-05-18T21:39:28.630 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2) ... 2020-05-18T21:39:28.871 INFO:teuthology.orchestra.run.smithi080.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-05-18T21:39:28.874 INFO:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-05-18T21:39:28.939 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2020-05-18T21:39:29.076 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2020-05-18T21:39:29.077 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2020-05-18T21:39:29.144 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up flex (2.6.4-6.2) ... 2020-05-18T21:39:29.205 INFO:teuthology.orchestra.run.smithi080.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2020-05-18T21:39:29.205 INFO:teuthology.orchestra.run.smithi080.stdout:libtool-bin is already the newest version (2.4.6-14). 2020-05-18T21:39:29.205 INFO:teuthology.orchestra.run.smithi080.stdout:network-manager is already the newest version (1.22.10-1ubuntu1). 2020-05-18T21:39:29.205 INFO:teuthology.orchestra.run.smithi080.stdout:network-manager set to manually installed. 2020-05-18T21:39:29.206 INFO:teuthology.orchestra.run.smithi080.stdout:util-linux is already the newest version (2.34-0.1ubuntu9). 2020-05-18T21:39:29.206 INFO:teuthology.orchestra.run.smithi080.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9). 2020-05-18T21:39:29.206 INFO:teuthology.orchestra.run.smithi080.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2020-05-18T21:39:29.206 INFO:teuthology.orchestra.run.smithi080.stdout:The following package was automatically installed and is no longer required: 2020-05-18T21:39:29.207 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.71.0 2020-05-18T21:39:29.207 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove it. 2020-05-18T21:39:29.207 INFO:teuthology.orchestra.run.smithi080.stdout:The following additional packages will be installed: 2020-05-18T21:39:29.207 INFO:teuthology.orchestra.run.smithi080.stdout: libfl-dev libfl2 liblzo2-2 2020-05-18T21:39:29.208 INFO:teuthology.orchestra.run.smithi080.stdout:Suggested packages: 2020-05-18T21:39:29.208 INFO:teuthology.orchestra.run.smithi080.stdout: bison-doc flex-doc indent-doc libssl-doc 2020-05-18T21:39:29.227 INFO:teuthology.orchestra.run.smithi080.stdout:The following NEW packages will be installed: 2020-05-18T21:39:29.228 INFO:teuthology.orchestra.run.smithi080.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2020-05-18T21:39:29.228 INFO:teuthology.orchestra.run.smithi080.stdout: libssl-dev 2020-05-18T21:39:29.268 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2020-05-18T21:39:29.394 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2020-05-18T21:39:29.406 INFO:teuthology.orchestra.run.smithi080.stdout:0 upgraded, 10 newly installed, 0 to remove and 45 not upgraded. 2020-05-18T21:39:29.406 INFO:teuthology.orchestra.run.smithi080.stdout:Need to get 2,912 kB of archives. 2020-05-18T21:39:29.407 INFO:teuthology.orchestra.run.smithi080.stdout:After this operation, 12.4 MB of additional disk space will be used. 2020-05-18T21:39:29.407 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2020-05-18T21:39:29.520 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2) ... 2020-05-18T21:39:29.646 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2020-05-18T21:39:29.813 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2020-05-18T21:39:29.873 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2020-05-18T21:39:29.939 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up dump (0.4b46-6) ... 2020-05-18T21:39:30.008 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2020-05-18T21:39:30.015 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2020-05-18T21:39:30.026 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-05-18T21:39:30.033 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2020-05-18T21:39:30.041 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2020-05-18T21:39:30.041 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2020-05-18T21:39:30.047 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2020-05-18T21:39:30.048 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2020-05-18T21:39:30.048 INFO:teuthology.orchestra.run.smithi080.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2 [1,582 kB] 2020-05-18T21:39:30.083 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2020-05-18T21:39:30.156 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-05-18T21:39:30.226 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up indent (2.2.12-1) ... 2020-05-18T21:39:30.352 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2020-05-18T21:39:30.479 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for libc-bin (2.31-0ubuntu9) ... 2020-05-18T21:39:30.535 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 2,912 kB in 1s (3,189 kB/s) 2020-05-18T21:39:30.655 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for man-db (2.9.1-1) ... 2020-05-18T21:39:30.714 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package flex. 2020-05-18T21:39:30.777 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 156462 files and directories currently installed.) 2020-05-18T21:39:30.782 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2020-05-18T21:39:30.828 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking flex (2.6.4-6.2) ... 2020-05-18T21:39:31.333 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package bison. 2020-05-18T21:39:31.351 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2020-05-18T21:39:31.396 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2020-05-18T21:39:31.838 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liblzo2-2:amd64. 2020-05-18T21:39:31.856 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2020-05-18T21:39:31.938 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2020-05-18T21:39:32.274 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package dump. 2020-05-18T21:39:32.294 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2020-05-18T21:39:32.333 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking dump (0.4b46-6) ... 2020-05-18T21:39:32.653 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package indent. 2020-05-18T21:39:32.671 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2020-05-18T21:39:32.711 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking indent (2.2.12-1) ... 2020-05-18T21:39:32.991 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2020-05-18T21:39:33.006 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-05-18T21:39:33.024 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2020-05-18T21:39:33.064 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2020-05-18T21:39:33.334 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-05-18T21:39:33.352 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2020-05-18T21:39:33.392 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2020-05-18T21:39:33.776 INFO:teuthology.orchestra.run.smithi196.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-05-18T21:39:33.779 DEBUG:teuthology.parallel:result is None 2020-05-18T21:39:33.788 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libfl2:amd64. 2020-05-18T21:39:33.805 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2020-05-18T21:39:33.846 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2020-05-18T21:39:34.160 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-05-18T21:39:34.179 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2020-05-18T21:39:34.442 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2020-05-18T21:39:35.010 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-05-18T21:39:35.030 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2_amd64.deb ... 2020-05-18T21:39:35.068 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2) ... 2020-05-18T21:39:35.533 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up flex (2.6.4-6.2) ... 2020-05-18T21:39:35.640 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2020-05-18T21:39:35.766 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2020-05-18T21:39:35.917 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2) ... 2020-05-18T21:39:36.043 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2020-05-18T21:39:36.178 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2020-05-18T21:39:36.304 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up dump (0.4b46-6) ... 2020-05-18T21:39:36.400 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-05-18T21:39:36.464 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2020-05-18T21:39:36.537 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-05-18T21:39:36.607 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up indent (2.2.12-1) ... 2020-05-18T21:39:36.733 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2020-05-18T21:39:36.895 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for libc-bin (2.31-0ubuntu9) ... 2020-05-18T21:39:37.070 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for man-db (2.9.1-1) ... 2020-05-18T21:39:39.323 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2020-05-18T21:39:40.216 INFO:teuthology.orchestra.run.smithi080.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-05-18T21:39:40.219 DEBUG:teuthology.parallel:result is None 2020-05-18T21:39:40.220 INFO:teuthology.orchestra.run.smithi196:> sudo lsb_release -is 2020-05-18T21:39:40.271 INFO:teuthology.orchestra.run.smithi196.stdout:Ubuntu 2020-05-18T21:39:40.276 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-05-18T21:39:40.277 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=543315c934420269aa12ef2f9dec2c9eadb4fa6f 2020-05-18T21:39:40.516 INFO:teuthology.orchestra.run.smithi196:> dpkg-query -W -f '${Version}' ceph 2020-05-18T21:39:40.542 INFO:teuthology.orchestra.run.smithi196.stdout:16.0.0-1640-g543315c9344-1focal 2020-05-18T21:39:40.543 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-1640-g543315c9344-1focal 2020-05-18T21:39:40.543 INFO:teuthology.task.install:The correct ceph version 16.0.0-1640-g543315c9344-1focal is installed. 2020-05-18T21:39:40.543 INFO:teuthology.orchestra.run.smithi080:> sudo lsb_release -is 2020-05-18T21:39:40.594 INFO:teuthology.orchestra.run.smithi080.stdout:Ubuntu 2020-05-18T21:39:40.599 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-05-18T21:39:40.600 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=543315c934420269aa12ef2f9dec2c9eadb4fa6f 2020-05-18T21:39:40.830 INFO:teuthology.orchestra.run.smithi080:> dpkg-query -W -f '${Version}' ceph 2020-05-18T21:39:40.855 INFO:teuthology.orchestra.run.smithi080.stdout:16.0.0-1640-g543315c9344-1focal 2020-05-18T21:39:40.856 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-1640-g543315c9344-1focal 2020-05-18T21:39:40.856 INFO:teuthology.task.install:The correct ceph version 16.0.0-1640-g543315c9344-1focal is installed. 2020-05-18T21:39:40.857 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-05-18T21:39:40.857 INFO:teuthology.orchestra.run.smithi196:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-05-18T21:39:40.875 INFO:teuthology.orchestra.run.smithi080:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-05-18T21:39:40.892 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-05-18T21:39:40.892 INFO:teuthology.orchestra.run.smithi196:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-05-18T21:39:40.909 INFO:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-05-18T21:39:40.923 INFO:teuthology.orchestra.run.smithi080:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-05-18T21:39:40.951 INFO:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-05-18T21:39:40.966 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-05-18T21:39:40.967 INFO:teuthology.orchestra.run.smithi196:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-05-18T21:39:40.983 INFO:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-05-18T21:39:40.996 INFO:teuthology.orchestra.run.smithi080:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-05-18T21:39:41.041 INFO:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-05-18T21:39:41.056 INFO:teuthology.run_tasks:Running task ceph... 2020-05-18T21:39:41.125 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-05-18T21:39:41.126 INFO:teuthology.orchestra.run.smithi080:> sudo chmod 777 /var/log/ceph 2020-05-18T21:39:41.128 INFO:teuthology.orchestra.run.smithi196:> sudo chmod 777 /var/log/ceph 2020-05-18T21:39:41.143 INFO:tasks.ceph:Disabling ceph logrotate... 2020-05-18T21:39:41.144 INFO:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/logrotate.d/ceph 2020-05-18T21:39:41.185 INFO:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/logrotate.d/ceph 2020-05-18T21:39:41.200 INFO:tasks.ceph:Creating extra log directories... 2020-05-18T21:39:41.201 INFO:teuthology.orchestra.run.smithi080:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-05-18T21:39:41.204 INFO:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-05-18T21:39:41.214 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2020-05-18T21:39:41.215 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2020-05-18T21:39:41.215 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2020-05-18T21:39:41.216 INFO:teuthology.orchestra.run.smithi196:> cat > /home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-05-18T21:39:41.264 INFO:teuthology.orchestra.run.smithi196:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-05-18T21:39:41.304 INFO:teuthology.orchestra.run.smithi080:> cat > /home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-05-18T21:39:41.311 INFO:teuthology.orchestra.run.smithi080:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-05-18T21:39:41.352 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-05-18T21:39:41.353 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd_max_omap_entries_per_request': 10000}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'SLOW_OPS', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-05-18T21:39:41.353 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2020-05-18_20:49:47-fs-wip-pdonnell-testing-20200516.030215-distro-basic-smithi/5067934', 'branch': 'wip-pdonnell-testing-20200516.030215', 'description': 'fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_ffsb.yaml}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '5067934', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2020-05-18_20:49:47-fs-wip-pdonnell-testing-20200516.030215-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20200516.030215'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd_max_omap_entries_per_request': 10000}}, 'fs': 'xfs', 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'SLOW_OPS', 'slow request'], 'sha1': '543315c934420269aa12ef2f9dec2c9eadb4fa6f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': '543315c934420269aa12ef2f9dec2c9eadb4fa6f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20200516.030215', 'sha1': '543315c934420269aa12ef2f9dec2c9eadb4fa6f'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'git://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'sha1': '543315c934420269aa12ef2f9dec2c9eadb4fa6f', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20200516.030215', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200516.030215/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '543315c934420269aa12ef2f9dec2c9eadb4fa6f', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '543315c934420269aa12ef2f9dec2c9eadb4fa6f'}}, {'ceph': None}, {'ceph-fuse': None}, {'check-counter': {'counters': {'mds': ['mds.dir_split']}}}, {'workunit': {'clients': {'all': ['suites/ffsb.sh']}}}], 'teuthology_branch': 'master', 'tube': 'smithi', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.30699', 'targets': {'smithi196.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXlKs/4Iub2TYjf6lRk+Xq/mWExXCpHbHiIDeEb2iyMCktl8JsJPbDS5GK1XVNP/tOrOCv+4Z8j3VigP/e17wirRvLgwXqdIU3neAxHgFVbjWomxnS74couN7s3gPpBtjC1Bp289v1Ofz8as1jjaj0lBdPUgR/l4N8alms7O9wR+wmWzrRbjiLAyOoaUcU0T91SqO6i/UEXNSgclq2vn9Gd2EX0QhoP6L+BP/xFSU3jAYRoLSI5Ls6uL3sP1PIF24nMP3EFoSFO8dvylVakjxSElH1O8EQcjBi7bd1dxfJdRFI93hwiDr66SarR8yLXHNX6/TvQjaQER/LLCmIedslZlHlc6vVyTCHhaY7+xmKfc2IdpdIKMVFyh9k2qpXi4N0edmOQziNpVMIbfsue0P9PmYcpNsrJ/qvWMGDPKtg8WwNPxUPvlQLXVc9Euj6mGH/K7uAkAy1gOjZOX86PzqtAXI2ITWF9QUJ/HeehvIuxip1Rp4Q7Hq59cSDGhzQ78c=', 'smithi080.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXqMrU7EmXoNhLeHD+zrdPTLqVqhGYFJLcBsbpiCRPDWWApstvKWCD452dNgsr0V+7LJZpP+DYjaDciuXYLvJIyTfi+E3jtNqpr3ypIsM13lJhYJYqVlX8B6VbfQWjnEAjVnOv3fgCiU14uEGAW2gSBq9FqAKoDXOxHHRRuLJhMJBDijDVJX9+1JjgxX31Nz1vQfMtfWpQLa7ii4n8w/ByL65WSTWg47lhKDnuzbs3jabP02mTziz5g7yoNlg3j0F7jT01L2Lbj3iYF08n0q7BiRB7/PAtrksm4XkVDoYeF+/YQ/EZgYx9vp4tZ7Ewx6CBAVxbgNkRvGRi4wkuDqCEMMIlM6Yex56eLlZ41OXooGJ+fREfFsIlA8+gAX4GUr/OZJtOjxbm4+o+y3WQaafccL6Uo4L/wcvssHPRWEyHju2ZvXUQSfeXvD6N23otMvqvD672rUumEAThDfjqcXp/Mu6FIe5Q0WFisWFe5bGEdN6mg8YFpqNjb0wO1VzyiuE='}} 2020-05-18T21:39:41.354 INFO:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-05-18T21:39:41.357 INFO:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-05-18T21:39:41.364 INFO:teuthology.orchestra.run.smithi080:> sudo install -d -m0777 -- /var/run/ceph 2020-05-18T21:39:41.405 INFO:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/run/ceph 2020-05-18T21:39:41.434 DEBUG:teuthology.orchestra.remote:smithi196:/scratch_devs is 72B 2020-05-18T21:39:41.448 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-05-18T21:39:41.448 INFO:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-05-18T21:39:41.454 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-05-18T21:39:41.455 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-18T21:39:41.455 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 6h/6d Inode: 346 Links: 1 2020-05-18T21:39:41.455 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-18T21:39:41.455 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2020-05-18 21:37:05.548433651 +0000 2020-05-18T21:39:41.456 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2020-05-18 21:37:05.352433653 +0000 2020-05-18T21:39:41.456 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2020-05-18 21:37:05.352433653 +0000 2020-05-18T21:39:41.456 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2020-05-18T21:39:41.464 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2020-05-18T21:39:41.464 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2020-05-18T21:39:41.465 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000428954 s, 1.2 MB/s 2020-05-18T21:39:41.469 INFO:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-05-18T21:39:41.516 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-05-18T21:39:41.517 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-18T21:39:41.517 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 6h/6d Inode: 351 Links: 1 2020-05-18T21:39:41.517 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-18T21:39:41.517 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2020-05-18 21:37:05.960433646 +0000 2020-05-18T21:39:41.517 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2020-05-18 21:37:05.800433648 +0000 2020-05-18T21:39:41.518 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2020-05-18 21:37:05.800433648 +0000 2020-05-18T21:39:41.518 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2020-05-18T21:39:41.527 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2020-05-18T21:39:41.527 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2020-05-18T21:39:41.528 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000349813 s, 1.5 MB/s 2020-05-18T21:39:41.531 INFO:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-05-18T21:39:41.538 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-05-18T21:39:41.538 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-18T21:39:41.538 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 6h/6d Inode: 358 Links: 1 2020-05-18T21:39:41.538 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-18T21:39:41.538 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2020-05-18 21:37:06.192433644 +0000 2020-05-18T21:39:41.539 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2020-05-18 21:37:06.188433644 +0000 2020-05-18T21:39:41.539 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2020-05-18 21:37:06.188433644 +0000 2020-05-18T21:39:41.539 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2020-05-18T21:39:41.546 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2020-05-18T21:39:41.547 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2020-05-18T21:39:41.547 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000229421 s, 2.2 MB/s 2020-05-18T21:39:41.551 INFO:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-05-18T21:39:41.557 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-05-18T21:39:41.558 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-18T21:39:41.558 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 6h/6d Inode: 366 Links: 1 2020-05-18T21:39:41.558 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-18T21:39:41.558 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2020-05-18 21:37:06.748433638 +0000 2020-05-18T21:39:41.558 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2020-05-18 21:37:06.584433639 +0000 2020-05-18T21:39:41.559 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2020-05-18 21:37:06.584433639 +0000 2020-05-18T21:39:41.559 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2020-05-18T21:39:41.566 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2020-05-18T21:39:41.567 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2020-05-18T21:39:41.567 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000353128 s, 1.4 MB/s 2020-05-18T21:39:41.571 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-05-18T21:39:41.583 DEBUG:teuthology.orchestra.remote:smithi080:/scratch_devs is 72B 2020-05-18T21:39:41.596 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-05-18T21:39:41.597 INFO:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-05-18T21:39:41.603 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-05-18T21:39:41.604 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-18T21:39:41.604 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 6h/6d Inode: 349 Links: 1 2020-05-18T21:39:41.604 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-18T21:39:41.604 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2020-05-18 21:37:04.670142452 +0000 2020-05-18T21:39:41.604 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2020-05-18 21:37:04.514142447 +0000 2020-05-18T21:39:41.605 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2020-05-18 21:37:04.514142447 +0000 2020-05-18T21:39:41.605 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2020-05-18T21:39:41.614 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2020-05-18T21:39:41.614 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2020-05-18T21:39:41.615 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000412291 s, 1.2 MB/s 2020-05-18T21:39:41.619 INFO:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-05-18T21:39:41.669 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-05-18T21:39:41.670 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-18T21:39:41.670 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 6h/6d Inode: 352 Links: 1 2020-05-18T21:39:41.670 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-18T21:39:41.670 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2020-05-18 21:37:05.050142465 +0000 2020-05-18T21:39:41.671 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2020-05-18 21:37:04.894142460 +0000 2020-05-18T21:39:41.671 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2020-05-18 21:37:04.894142460 +0000 2020-05-18T21:39:41.671 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2020-05-18T21:39:41.680 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2020-05-18T21:39:41.681 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2020-05-18T21:39:41.681 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000358531 s, 1.4 MB/s 2020-05-18T21:39:41.685 INFO:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-05-18T21:39:41.692 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-05-18T21:39:41.692 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-18T21:39:41.692 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 6h/6d Inode: 361 Links: 1 2020-05-18T21:39:41.693 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-18T21:39:41.693 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2020-05-18 21:37:05.454142478 +0000 2020-05-18T21:39:41.693 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2020-05-18 21:37:05.298142473 +0000 2020-05-18T21:39:41.693 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2020-05-18 21:37:05.298142473 +0000 2020-05-18T21:39:41.693 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2020-05-18T21:39:41.699 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2020-05-18T21:39:41.700 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2020-05-18T21:39:41.700 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000358278 s, 1.4 MB/s 2020-05-18T21:39:41.704 INFO:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-05-18T21:39:41.711 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-05-18T21:39:41.711 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-18T21:39:41.711 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 6h/6d Inode: 368 Links: 1 2020-05-18T21:39:41.711 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-18T21:39:41.711 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2020-05-18 21:37:05.690142485 +0000 2020-05-18T21:39:41.712 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2020-05-18 21:37:05.686142485 +0000 2020-05-18T21:39:41.712 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2020-05-18 21:37:05.686142485 +0000 2020-05-18T21:39:41.712 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2020-05-18T21:39:41.721 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2020-05-18T21:39:41.721 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2020-05-18T21:39:41.722 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000285737 s, 1.8 MB/s 2020-05-18T21:39:41.726 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-05-18T21:39:41.726 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi080.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-05-18T21:39:41.727 INFO:tasks.ceph:Generating config... 2020-05-18T21:39:41.728 INFO:tasks.ceph:[client] client mount timeout = 600 2020-05-18T21:39:41.728 INFO:tasks.ceph:[client] debug client = 20 2020-05-18T21:39:41.729 INFO:tasks.ceph:[client] debug ms = 1 2020-05-18T21:39:41.729 INFO:tasks.ceph:[mds] debug mds = 20 2020-05-18T21:39:41.729 INFO:tasks.ceph:[mds] debug ms = 1 2020-05-18T21:39:41.730 INFO:tasks.ceph:[mds] mds bal frag = True 2020-05-18T21:39:41.730 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2020-05-18T21:39:41.730 INFO:tasks.ceph:[mds] mds bal merge size = 5 2020-05-18T21:39:41.730 INFO:tasks.ceph:[mds] mds bal split bits = 3 2020-05-18T21:39:41.731 INFO:tasks.ceph:[mds] mds bal split size = 100 2020-05-18T21:39:41.731 INFO:tasks.ceph:[mds] mds debug frag = True 2020-05-18T21:39:41.731 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2020-05-18T21:39:41.732 INFO:tasks.ceph:[mds] mds op complaint time = 180 2020-05-18T21:39:41.732 INFO:tasks.ceph:[mds] mds verify scatter = True 2020-05-18T21:39:41.732 INFO:tasks.ceph:[mds] osd op complaint time = 180 2020-05-18T21:39:41.732 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-05-18T21:39:41.733 INFO:tasks.ceph:[mgr] debug ms = 1 2020-05-18T21:39:41.733 INFO:tasks.ceph:[mon] debug mon = 20 2020-05-18T21:39:41.733 INFO:tasks.ceph:[mon] debug ms = 1 2020-05-18T21:39:41.733 INFO:tasks.ceph:[mon] debug paxos = 20 2020-05-18T21:39:41.734 INFO:tasks.ceph:[mon] mon op complaint time = 120 2020-05-18T21:39:41.734 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-05-18T21:39:41.734 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2020-05-18T21:39:41.735 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-05-18T21:39:41.735 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-05-18T21:39:41.735 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-05-18T21:39:41.735 INFO:tasks.ceph:[osd] debug filestore = 20 2020-05-18T21:39:41.736 INFO:tasks.ceph:[osd] debug journal = 20 2020-05-18T21:39:41.736 INFO:tasks.ceph:[osd] debug ms = 20 2020-05-18T21:39:41.736 INFO:tasks.ceph:[osd] debug osd = 25 2020-05-18T21:39:41.736 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-05-18T21:39:41.737 INFO:tasks.ceph:[osd] filestore flush min = 0 2020-05-18T21:39:41.737 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-05-18T21:39:41.737 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-05-18T21:39:41.738 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-05-18T21:39:41.738 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-05-18T21:39:41.738 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-05-18T21:39:41.738 INFO:tasks.ceph:[osd] osd op complaint time = 180 2020-05-18T21:39:41.739 INFO:tasks.ceph:[osd] osd_max_omap_entries_per_request = 10000 2020-05-18T21:39:41.739 INFO:tasks.ceph:Setting up mon.a... 2020-05-18T21:39:41.739 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-05-18T21:39:41.783 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.keyring 2020-05-18T21:39:41.786 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-05-18T21:39:41.864 INFO:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-05-18T21:39:41.879 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.196'), ('mon.b', '172.21.15.80'), ('mon.c', '[v2:172.21.15.80:3301,v1:172.21.15.80:6790]')] 2020-05-18T21:39:41.880 INFO:teuthology.orchestra.run.smithi196:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.196 --add b 172.21.15.80 --addv c '[v2:172.21.15.80:3301,v1:172.21.15.80:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:39:41.913 INFO:teuthology.orchestra.run.smithi196.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:39:41.913 INFO:teuthology.orchestra.run.smithi196.stdout:monmaptool: generated fsid d048673a-3d91-4cc6-882d-e9cc70e7cd6c 2020-05-18T21:39:41.914 INFO:teuthology.orchestra.run.smithi196.stdout:epoch 0 2020-05-18T21:39:41.914 INFO:teuthology.orchestra.run.smithi196.stdout:fsid d048673a-3d91-4cc6-882d-e9cc70e7cd6c 2020-05-18T21:39:41.914 INFO:teuthology.orchestra.run.smithi196.stdout:last_changed 2020-05-18T21:39:41.917255+0000 2020-05-18T21:39:41.914 INFO:teuthology.orchestra.run.smithi196.stdout:created 2020-05-18T21:39:41.917255+0000 2020-05-18T21:39:41.914 INFO:teuthology.orchestra.run.smithi196.stdout:min_mon_release 0 (unknown) 2020-05-18T21:39:41.915 INFO:teuthology.orchestra.run.smithi196.stdout:0: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.a 2020-05-18T21:39:41.915 INFO:teuthology.orchestra.run.smithi196.stdout:1: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.b 2020-05-18T21:39:41.915 INFO:teuthology.orchestra.run.smithi196.stdout:2: [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] mon.c 2020-05-18T21:39:41.915 INFO:teuthology.orchestra.run.smithi196.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2020-05-18T21:39:41.916 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d048673a-3d91-4cc6-882d-e9cc70e7cd6c... 2020-05-18T21:39:41.917 INFO:teuthology.orchestra.run.smithi080:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-05-18T21:39:41.920 INFO:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-05-18T21:39:41.950 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2020-05-18T21:39:41.950 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2020-05-18T21:39:41.951 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-05-18T21:39:41.951 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2020-05-18T21:39:41.951 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.951 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2020-05-18T21:39:41.951 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.952 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2020-05-18T21:39:41.952 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.952 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2020-05-18T21:39:41.952 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2020-05-18T21:39:41.952 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.953 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2020-05-18T21:39:41.953 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2020-05-18T21:39:41.953 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.953 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2020-05-18T21:39:41.953 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2020-05-18T21:39:41.954 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.954 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2020-05-18T21:39:41.954 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2020-05-18T21:39:41.954 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.954 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2020-05-18T21:39:41.954 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.955 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2020-05-18T21:39:41.955 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2020-05-18T21:39:41.955 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2020-05-18T21:39:41.955 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2020-05-18T21:39:41.955 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2020-05-18T21:39:41.956 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2020-05-18T21:39:41.956 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2020-05-18T21:39:41.956 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-05-18T21:39:41.956 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2020-05-18T21:39:41.956 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2020-05-18T21:39:41.956 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.957 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-05-18T21:39:41.957 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.957 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2020-05-18T21:39:41.957 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.958 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2020-05-18T21:39:41.958 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.958 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2020-05-18T21:39:41.958 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2020-05-18T21:39:41.958 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.196,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2020-05-18T21:39:41.959 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = d048673a-3d91-4cc6-882d-e9cc70e7cd6c 2020-05-18T21:39:41.959 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.959 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2020-05-18T21:39:41.959 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2020-05-18T21:39:41.959 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.959 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2020-05-18T21:39:41.960 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2020-05-18T21:39:41.960 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.960 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2020-05-18T21:39:41.960 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2020-05-18T21:39:41.960 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.961 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2020-05-18T21:39:41.961 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2020-05-18T21:39:41.961 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2020-05-18T21:39:41.961 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.961 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2020-05-18T21:39:41.961 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2020-05-18T21:39:41.962 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.962 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2020-05-18T21:39:41.962 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.962 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2020-05-18T21:39:41.962 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.963 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2020-05-18T21:39:41.963 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.963 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2020-05-18T21:39:41.963 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2020-05-18T21:39:41.963 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.963 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2020-05-18T21:39:41.964 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2020-05-18T21:39:41.964 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2020-05-18T21:39:41.964 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore compression mode = aggressive 2020-05-18T21:39:41.964 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2020-05-18T21:39:41.964 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 20 2020-05-18T21:39:41.965 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 20 2020-05-18T21:39:41.965 INFO:teuthology.orchestra.run.smithi196.stdout: debug filestore = 20 2020-05-18T21:39:41.965 INFO:teuthology.orchestra.run.smithi196.stdout: debug journal = 20 2020-05-18T21:39:41.965 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 20 2020-05-18T21:39:41.965 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 25 2020-05-18T21:39:41.965 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 10 2020-05-18T21:39:41.966 INFO:teuthology.orchestra.run.smithi196.stdout: filestore flush min = 0 2020-05-18T21:39:41.966 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2020-05-18T21:39:41.966 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2020-05-18T21:39:41.966 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2020-05-18T21:39:41.966 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2020-05-18T21:39:41.967 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2020-05-18T21:39:41.967 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2020-05-18T21:39:41.967 INFO:teuthology.orchestra.run.smithi196.stdout: osd_max_omap_entries_per_request = 10000 2020-05-18T21:39:41.967 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.967 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2020-05-18T21:39:41.967 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2020-05-18T21:39:41.968 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2020-05-18T21:39:41.968 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2020-05-18T21:39:41.968 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2020-05-18T21:39:41.968 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2020-05-18T21:39:41.968 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2020-05-18T21:39:41.968 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2020-05-18T21:39:41.969 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.969 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2020-05-18T21:39:41.969 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2020-05-18T21:39:41.969 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2020-05-18T21:39:41.969 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2020-05-18T21:39:41.970 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2020-05-18T21:39:41.970 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2020-05-18T21:39:41.970 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2020-05-18T21:39:41.970 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2020-05-18T21:39:41.970 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2020-05-18T21:39:41.970 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2020-05-18T21:39:41.971 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2020-05-18T21:39:41.971 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2020-05-18T21:39:41.971 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:39:41.971 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2020-05-18T21:39:41.971 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2020-05-18T21:39:41.972 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2020-05-18T21:39:41.972 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2020-05-18T21:39:41.972 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-05-18T21:39:41.972 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-05-18T21:39:41.972 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2020-05-18T21:39:41.972 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2020-05-18T21:39:41.973 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2020-05-18T21:39:41.973 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2020-05-18T21:39:41.973 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2020-05-18T21:39:41.973 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2020-05-18T21:39:41.973 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2020-05-18T21:39:41.974 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2020-05-18T21:39:41.974 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2020-05-18T21:39:41.974 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2020-05-18T21:39:41.974 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2020-05-18T21:39:41.974 INFO:teuthology.orchestra.run.smithi196.stdout: mds bal frag = True 2020-05-18T21:39:41.974 INFO:teuthology.orchestra.run.smithi196.stdout: mds bal fragment size max = 10000 2020-05-18T21:39:41.975 INFO:teuthology.orchestra.run.smithi196.stdout: mds bal merge size = 5 2020-05-18T21:39:41.975 INFO:teuthology.orchestra.run.smithi196.stdout: mds bal split bits = 3 2020-05-18T21:39:41.975 INFO:teuthology.orchestra.run.smithi196.stdout: mds bal split size = 100 2020-05-18T21:39:41.975 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2020-05-18T21:39:41.975 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = True 2020-05-18T21:39:41.975 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2020-05-18T21:39:41.976 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = True 2020-05-18T21:39:41.976 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2020-05-18T21:39:41.977 INFO:teuthology.orchestra.run.smithi080.stdout:[global] 2020-05-18T21:39:41.978 INFO:teuthology.orchestra.run.smithi080.stdout: chdir = "" 2020-05-18T21:39:41.978 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-05-18T21:39:41.978 INFO:teuthology.orchestra.run.smithi080.stdout: auth supported = cephx 2020-05-18T21:39:41.978 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.978 INFO:teuthology.orchestra.run.smithi080.stdout: filestore xattr use omap = true 2020-05-18T21:39:41.979 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.979 INFO:teuthology.orchestra.run.smithi080.stdout: mon clock drift allowed = 1.000 2020-05-18T21:39:41.979 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.979 INFO:teuthology.orchestra.run.smithi080.stdout: osd crush chooseleaf type = 0 2020-05-18T21:39:41.979 INFO:teuthology.orchestra.run.smithi080.stdout: auth debug = true 2020-05-18T21:39:41.979 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.980 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on old message = true 2020-05-18T21:39:41.980 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on bug = true 2020-05-18T21:39:41.980 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.980 INFO:teuthology.orchestra.run.smithi080.stdout: mon max pg per osd = 10000 # >= luminous 2020-05-18T21:39:41.980 INFO:teuthology.orchestra.run.smithi080.stdout: mon pg warn max object skew = 0 2020-05-18T21:39:41.981 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.981 INFO:teuthology.orchestra.run.smithi080.stdout: # disable pg_autoscaler by default for new pools 2020-05-18T21:39:41.981 INFO:teuthology.orchestra.run.smithi080.stdout: osd_pool_default_pg_autoscale_mode = off 2020-05-18T21:39:41.981 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.981 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default size = 2 2020-05-18T21:39:41.982 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.982 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow primary affinity = true 2020-05-18T21:39:41.982 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow pg remap = true 2020-05-18T21:39:41.982 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on legacy crush tunables = false 2020-05-18T21:39:41.982 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on crush straw calc version zero = false 2020-05-18T21:39:41.982 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on no sortbitwise = false 2020-05-18T21:39:41.983 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on osd down out interval zero = false 2020-05-18T21:39:41.983 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on too few osds = false 2020-05-18T21:39:41.983 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-05-18T21:39:41.983 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_no_redundancy = false 2020-05-18T21:39:41.983 INFO:teuthology.orchestra.run.smithi080.stdout: mon_allow_pool_size_one = true 2020-05-18T21:39:41.984 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.984 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-05-18T21:39:41.984 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.984 INFO:teuthology.orchestra.run.smithi080.stdout: osd default data pool replay window = 5 2020-05-18T21:39:41.984 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.984 INFO:teuthology.orchestra.run.smithi080.stdout: mon allow pool delete = true 2020-05-18T21:39:41.985 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.985 INFO:teuthology.orchestra.run.smithi080.stdout: mon cluster log file level = debug 2020-05-18T21:39:41.985 INFO:teuthology.orchestra.run.smithi080.stdout: debug asserts on shutdown = true 2020-05-18T21:39:41.985 INFO:teuthology.orchestra.run.smithi080.stdout: mon host = "172.21.15.196,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2020-05-18T21:39:41.985 INFO:teuthology.orchestra.run.smithi080.stdout: fsid = d048673a-3d91-4cc6-882d-e9cc70e7cd6c 2020-05-18T21:39:41.986 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.986 INFO:teuthology.orchestra.run.smithi080.stdout:[osd] 2020-05-18T21:39:41.986 INFO:teuthology.orchestra.run.smithi080.stdout: osd journal size = 100 2020-05-18T21:39:41.986 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.986 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub load threshold = 5.0 2020-05-18T21:39:41.986 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub max interval = 600 2020-05-18T21:39:41.987 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.987 INFO:teuthology.orchestra.run.smithi080.stdout: osd recover clone overlap = true 2020-05-18T21:39:41.987 INFO:teuthology.orchestra.run.smithi080.stdout: osd recovery max chunk = 1048576 2020-05-18T21:39:41.987 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.987 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug shutdown = true 2020-05-18T21:39:41.988 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug op order = true 2020-05-18T21:39:41.988 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug verify stray on activate = true 2020-05-18T21:39:41.988 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.988 INFO:teuthology.orchestra.run.smithi080.stdout: osd open classes on start = true 2020-05-18T21:39:41.988 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug pg log writeout = true 2020-05-18T21:39:41.988 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.989 INFO:teuthology.orchestra.run.smithi080.stdout: osd deep scrub update digest min age = 30 2020-05-18T21:39:41.989 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.989 INFO:teuthology.orchestra.run.smithi080.stdout: osd map max advance = 10 2020-05-18T21:39:41.989 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.989 INFO:teuthology.orchestra.run.smithi080.stdout: journal zero on create = true 2020-05-18T21:39:41.990 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.990 INFO:teuthology.orchestra.run.smithi080.stdout: filestore ondisk finisher threads = 3 2020-05-18T21:39:41.990 INFO:teuthology.orchestra.run.smithi080.stdout: filestore apply finisher threads = 3 2020-05-18T21:39:41.990 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.990 INFO:teuthology.orchestra.run.smithi080.stdout: bdev debug aio = true 2020-05-18T21:39:41.990 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug misdirected ops = true 2020-05-18T21:39:41.991 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore block size = 96636764160 2020-05-18T21:39:41.991 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore compression mode = aggressive 2020-05-18T21:39:41.991 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore fsck on mount = True 2020-05-18T21:39:41.991 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluefs = 20 2020-05-18T21:39:41.991 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluestore = 20 2020-05-18T21:39:41.992 INFO:teuthology.orchestra.run.smithi080.stdout: debug filestore = 20 2020-05-18T21:39:41.992 INFO:teuthology.orchestra.run.smithi080.stdout: debug journal = 20 2020-05-18T21:39:41.992 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 20 2020-05-18T21:39:41.992 INFO:teuthology.orchestra.run.smithi080.stdout: debug osd = 25 2020-05-18T21:39:41.992 INFO:teuthology.orchestra.run.smithi080.stdout: debug rocksdb = 10 2020-05-18T21:39:41.992 INFO:teuthology.orchestra.run.smithi080.stdout: filestore flush min = 0 2020-05-18T21:39:41.993 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd backfillfull_ratio = 0.85 2020-05-18T21:39:41.993 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd full ratio = 0.9 2020-05-18T21:39:41.993 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd nearfull ratio = 0.8 2020-05-18T21:39:41.993 INFO:teuthology.orchestra.run.smithi080.stdout: osd failsafe full ratio = 0.95 2020-05-18T21:39:41.993 INFO:teuthology.orchestra.run.smithi080.stdout: osd objectstore = bluestore 2020-05-18T21:39:41.993 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2020-05-18T21:39:41.994 INFO:teuthology.orchestra.run.smithi080.stdout: osd_max_omap_entries_per_request = 10000 2020-05-18T21:39:41.994 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.994 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr] 2020-05-18T21:39:41.994 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2020-05-18T21:39:41.994 INFO:teuthology.orchestra.run.smithi080.stdout: debug mgr = 20 2020-05-18T21:39:41.995 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2020-05-18T21:39:41.995 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2020-05-18T21:39:41.995 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2020-05-18T21:39:41.995 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2020-05-18T21:39:41.995 INFO:teuthology.orchestra.run.smithi080.stdout: mgr/telemetry/nag = false 2020-05-18T21:39:41.995 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.996 INFO:teuthology.orchestra.run.smithi080.stdout:[mon] 2020-05-18T21:39:41.996 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2020-05-18T21:39:41.996 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2020-05-18T21:39:41.996 INFO:teuthology.orchestra.run.smithi080.stdout: debug paxos = 20 2020-05-18T21:39:41.996 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2020-05-18T21:39:41.997 INFO:teuthology.orchestra.run.smithi080.stdout: mon data avail warn = 5 2020-05-18T21:39:41.997 INFO:teuthology.orchestra.run.smithi080.stdout: mon mgr mkfs grace = 240 2020-05-18T21:39:41.997 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2020-05-18T21:39:41.997 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd reporter subtree level = osd 2020-05-18T21:39:41.997 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd prime pg temp = true 2020-05-18T21:39:41.997 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2020-05-18T21:39:41.998 INFO:teuthology.orchestra.run.smithi080.stdout: mon op complaint time = 120 2020-05-18T21:39:41.998 INFO:teuthology.orchestra.run.smithi080.stdout: 2020-05-18T21:39:41.998 INFO:teuthology.orchestra.run.smithi080.stdout:[client] 2020-05-18T21:39:41.998 INFO:teuthology.orchestra.run.smithi080.stdout: rgw cache enabled = true 2020-05-18T21:39:41.998 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable ops log = true 2020-05-18T21:39:41.999 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable usage log = true 2020-05-18T21:39:41.999 INFO:teuthology.orchestra.run.smithi080.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-05-18T21:39:41.999 INFO:teuthology.orchestra.run.smithi080.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-05-18T21:39:41.999 INFO:teuthology.orchestra.run.smithi080.stdout: client mount timeout = 600 2020-05-18T21:39:41.999 INFO:teuthology.orchestra.run.smithi080.stdout: debug client = 20 2020-05-18T21:39:41.999 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2020-05-18T21:39:42.000 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.a] 2020-05-18T21:39:42.000 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.b] 2020-05-18T21:39:42.000 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.c] 2020-05-18T21:39:42.000 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.a] 2020-05-18T21:39:42.000 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.b] 2020-05-18T21:39:42.001 INFO:teuthology.orchestra.run.smithi080.stdout:[mds] 2020-05-18T21:39:42.001 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds = 20 2020-05-18T21:39:42.001 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2020-05-18T21:39:42.001 INFO:teuthology.orchestra.run.smithi080.stdout: mds bal frag = True 2020-05-18T21:39:42.001 INFO:teuthology.orchestra.run.smithi080.stdout: mds bal fragment size max = 10000 2020-05-18T21:39:42.001 INFO:teuthology.orchestra.run.smithi080.stdout: mds bal merge size = 5 2020-05-18T21:39:42.002 INFO:teuthology.orchestra.run.smithi080.stdout: mds bal split bits = 3 2020-05-18T21:39:42.002 INFO:teuthology.orchestra.run.smithi080.stdout: mds bal split size = 100 2020-05-18T21:39:42.002 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug frag = True 2020-05-18T21:39:42.002 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug scatterstat = True 2020-05-18T21:39:42.002 INFO:teuthology.orchestra.run.smithi080.stdout: mds op complaint time = 180 2020-05-18T21:39:42.003 INFO:teuthology.orchestra.run.smithi080.stdout: mds verify scatter = True 2020-05-18T21:39:42.003 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2020-05-18T21:39:42.003 INFO:tasks.ceph:Creating admin key on mon.a... 2020-05-18T21:39:42.004 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-05-18T21:39:42.062 INFO:tasks.ceph:Copying monmap to all nodes... 2020-05-18T21:39:42.073 DEBUG:teuthology.orchestra.remote:smithi196:/etc/ceph/ceph.keyring is 206B 2020-05-18T21:39:42.091 DEBUG:teuthology.orchestra.remote:smithi196:/home/ubuntu/cephtest/ceph.monmap is 379B 2020-05-18T21:39:42.103 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi196.front.sepia.ceph.com 2020-05-18T21:39:42.104 INFO:teuthology.orchestra.run.smithi196:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2020-05-18T21:39:42.131 INFO:teuthology.orchestra.run.smithi196:> cat > /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:39:42.181 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi080.front.sepia.ceph.com 2020-05-18T21:39:42.181 INFO:teuthology.orchestra.run.smithi080:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2020-05-18T21:39:42.210 INFO:teuthology.orchestra.run.smithi080:> cat > /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:39:42.256 INFO:tasks.ceph:Setting up mon nodes... 2020-05-18T21:39:42.257 INFO:tasks.ceph:Setting up mgr nodes... 2020-05-18T21:39:42.257 INFO:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2020-05-18T21:39:42.300 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2020-05-18T21:39:42.302 INFO:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-05-18T21:39:42.356 INFO:teuthology.orchestra.run.smithi080.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-05-18T21:39:42.359 INFO:tasks.ceph:Setting up mds nodes... 2020-05-18T21:39:42.360 INFO:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2020-05-18T21:39:42.403 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2020-05-18T21:39:42.405 INFO:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2020-05-18T21:39:42.462 INFO:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2020-05-18T21:39:42.504 INFO:teuthology.orchestra.run.smithi080.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2020-05-18T21:39:42.507 INFO:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2020-05-18T21:39:42.567 INFO:tasks.ceph_client:Setting up client nodes... 2020-05-18T21:39:42.567 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-05-18T21:39:42.602 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-05-18T21:39:42.615 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-05-18T21:39:42.616 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi080.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-05-18T21:39:42.616 INFO:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-05-18T21:39:42.674 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-05-18T21:39:42.674 INFO:tasks.ceph:role: osd.0 2020-05-18T21:39:42.675 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com 2020-05-18T21:39:42.675 INFO:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-05-18T21:39:43.198 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-05-18T21:39:43.198 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-18T21:39:43.198 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-05-18T21:39:43.199 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 2020-05-18T21:39:43.199 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-18T21:39:43.199 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2020-05-18T21:39:43.199 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-05-18T21:39:43.199 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-18T21:39:43.200 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-18T21:39:43.200 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-18T21:39:43.201 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2020-05-18T21:39:43.201 INFO:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-05-18T21:39:43.292 INFO:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-05-18T21:39:43.303 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2020-05-18T21:39:43.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-05-18T21:39:43.305 INFO:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-05-18T21:39:43.321 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-05-18T21:39:43.321 INFO:tasks.ceph:role: osd.1 2020-05-18T21:39:43.321 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi196.front.sepia.ceph.com 2020-05-18T21:39:43.322 INFO:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-05-18T21:39:43.837 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-05-18T21:39:43.838 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-18T21:39:43.838 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-05-18T21:39:43.838 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 2020-05-18T21:39:43.838 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-18T21:39:43.839 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2020-05-18T21:39:43.839 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-05-18T21:39:43.839 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-18T21:39:43.839 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-18T21:39:43.839 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-18T21:39:43.840 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2020-05-18T21:39:43.841 INFO:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2020-05-18T21:39:43.886 INFO:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-05-18T21:39:43.899 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2020-05-18T21:39:43.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-05-18T21:39:43.900 INFO:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-05-18T21:39:43.913 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-05-18T21:39:43.914 INFO:tasks.ceph:role: osd.2 2020-05-18T21:39:43.914 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi196.front.sepia.ceph.com 2020-05-18T21:39:43.914 INFO:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-05-18T21:39:44.446 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-05-18T21:39:44.447 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-18T21:39:44.447 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-05-18T21:39:44.447 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 2020-05-18T21:39:44.448 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-18T21:39:44.448 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2020-05-18T21:39:44.448 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-05-18T21:39:44.448 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-18T21:39:44.448 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-18T21:39:44.448 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-18T21:39:44.449 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2020-05-18T21:39:44.450 INFO:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2020-05-18T21:39:44.503 INFO:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-05-18T21:39:44.514 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2020-05-18T21:39:44.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-05-18T21:39:44.516 INFO:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2020-05-18T21:39:44.529 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-05-18T21:39:44.529 INFO:tasks.ceph:role: osd.3 2020-05-18T21:39:44.530 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi196.front.sepia.ceph.com 2020-05-18T21:39:44.530 INFO:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-05-18T21:39:45.086 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-05-18T21:39:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-18T21:39:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-05-18T21:39:45.086 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 2020-05-18T21:39:45.087 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-18T21:39:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2020-05-18T21:39:45.087 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-05-18T21:39:45.087 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-18T21:39:45.087 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-18T21:39:45.088 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-18T21:39:45.088 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2020-05-18T21:39:45.089 INFO:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2020-05-18T21:39:45.149 INFO:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2020-05-18T21:39:45.160 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2020-05-18T21:39:45.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-05-18T21:39:45.162 INFO:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:39:45.264 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:45.263+0000 7ff71e94cdc0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-05-18T21:39:45.266 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:45.267+0000 7ff71e94cdc0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-05-18T21:39:45.267 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:45.267+0000 7ff71e94cdc0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-05-18T21:39:45.267 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:45.267+0000 7ff71e94cdc0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-05-18T21:39:45.267 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:45.267+0000 7ff71e94cdc0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-05-18T21:39:45.275 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:45.275+0000 7ff71e94cdc0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-05-18T21:39:47.753 INFO:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-05-18T21:39:47.770 INFO:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:39:47.805 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:47.803+0000 7ff4774d9dc0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-05-18T21:39:47.806 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:47.803+0000 7ff4774d9dc0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-05-18T21:39:47.807 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:47.803+0000 7ff4774d9dc0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-05-18T21:39:47.807 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:47.803+0000 7ff4774d9dc0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-05-18T21:39:47.807 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:47.803+0000 7ff4774d9dc0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-05-18T21:39:47.807 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:47.807+0000 7ff4774d9dc0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-05-18T21:39:50.268 INFO:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-05-18T21:39:50.326 INFO:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:39:50.360 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:50.359+0000 7f9951bb8dc0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-05-18T21:39:50.361 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:50.359+0000 7f9951bb8dc0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-05-18T21:39:50.361 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:50.359+0000 7f9951bb8dc0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-05-18T21:39:50.361 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:50.359+0000 7f9951bb8dc0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-05-18T21:39:50.361 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:50.359+0000 7f9951bb8dc0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-05-18T21:39:50.362 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:50.359+0000 7f9951bb8dc0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-05-18T21:39:52.820 INFO:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-05-18T21:39:52.837 INFO:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:39:52.877 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:52.875+0000 7f2ca8db6dc0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2020-05-18T21:39:52.877 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:52.875+0000 7f2ca8db6dc0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2020-05-18T21:39:52.878 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:52.875+0000 7f2ca8db6dc0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-05-18T21:39:52.878 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:52.875+0000 7f2ca8db6dc0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-05-18T21:39:52.878 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:52.875+0000 7f2ca8db6dc0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-05-18T21:39:52.879 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:39:52.879+0000 7f2ca8db6dc0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2020-05-18T21:39:55.344 INFO:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-05-18T21:39:55.402 INFO:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2020-05-18T21:39:55.419 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-05-18T21:39:55.419 INFO:tasks.ceph:role: osd.4 2020-05-18T21:39:55.420 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi080.front.sepia.ceph.com 2020-05-18T21:39:55.420 INFO:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-05-18T21:39:55.945 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-05-18T21:39:55.945 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-18T21:39:55.946 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-05-18T21:39:55.946 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 2020-05-18T21:39:55.946 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-18T21:39:55.946 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2020-05-18T21:39:55.946 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-05-18T21:39:55.946 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-18T21:39:55.947 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-18T21:39:55.947 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-18T21:39:55.948 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2020-05-18T21:39:55.948 INFO:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2020-05-18T21:39:56.039 INFO:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2020-05-18T21:39:56.052 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2020-05-18T21:39:56.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-05-18T21:39:56.054 INFO:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2020-05-18T21:39:56.068 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-05-18T21:39:56.069 INFO:tasks.ceph:role: osd.5 2020-05-18T21:39:56.069 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi080.front.sepia.ceph.com 2020-05-18T21:39:56.069 INFO:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-05-18T21:39:56.592 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-05-18T21:39:56.592 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-18T21:39:56.592 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-05-18T21:39:56.593 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 2020-05-18T21:39:56.593 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-18T21:39:56.593 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2020-05-18T21:39:56.593 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-05-18T21:39:56.593 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-18T21:39:56.594 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-18T21:39:56.594 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-18T21:39:56.595 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2020-05-18T21:39:56.595 INFO:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2020-05-18T21:39:56.647 INFO:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2020-05-18T21:39:56.658 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2020-05-18T21:39:56.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-05-18T21:39:56.659 INFO:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2020-05-18T21:39:56.669 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-05-18T21:39:56.669 INFO:tasks.ceph:role: osd.6 2020-05-18T21:39:56.669 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi080.front.sepia.ceph.com 2020-05-18T21:39:56.670 INFO:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-05-18T21:39:57.123 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-05-18T21:39:57.123 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-18T21:39:57.124 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-05-18T21:39:57.124 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 2020-05-18T21:39:57.124 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-18T21:39:57.124 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2020-05-18T21:39:57.124 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-05-18T21:39:57.124 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-18T21:39:57.125 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-18T21:39:57.125 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-18T21:39:57.126 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2020-05-18T21:39:57.126 INFO:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2020-05-18T21:39:57.188 INFO:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2020-05-18T21:39:57.200 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2020-05-18T21:39:57.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-05-18T21:39:57.201 INFO:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2020-05-18T21:39:57.211 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-05-18T21:39:57.211 INFO:tasks.ceph:role: osd.7 2020-05-18T21:39:57.212 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi080.front.sepia.ceph.com 2020-05-18T21:39:57.212 INFO:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-05-18T21:39:57.705 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-05-18T21:39:57.705 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-18T21:39:57.705 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-05-18T21:39:57.706 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 2020-05-18T21:39:57.706 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-18T21:39:57.706 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2020-05-18T21:39:57.706 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-05-18T21:39:57.706 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-18T21:39:57.707 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-18T21:39:57.707 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-18T21:39:57.708 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2020-05-18T21:39:57.708 INFO:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2020-05-18T21:39:57.776 INFO:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2020-05-18T21:39:57.785 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2020-05-18T21:39:57.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-05-18T21:39:57.786 INFO:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:39:57.887 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:39:57.885+0000 7fc2b9698dc0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2020-05-18T21:39:57.888 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:39:57.885+0000 7fc2b9698dc0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2020-05-18T21:39:57.889 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:39:57.885+0000 7fc2b9698dc0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-05-18T21:39:57.889 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:39:57.885+0000 7fc2b9698dc0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-05-18T21:39:57.889 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:39:57.885+0000 7fc2b9698dc0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-05-18T21:39:57.890 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:39:57.885+0000 7fc2b9698dc0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2020-05-18T21:40:00.361 INFO:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-05-18T21:40:00.378 INFO:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:40:00.415 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:00.409+0000 7f8ebc2f7dc0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2020-05-18T21:40:00.416 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:00.409+0000 7f8ebc2f7dc0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2020-05-18T21:40:00.416 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:00.409+0000 7f8ebc2f7dc0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-05-18T21:40:00.416 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:00.409+0000 7f8ebc2f7dc0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-05-18T21:40:00.416 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:00.409+0000 7f8ebc2f7dc0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-05-18T21:40:00.417 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:00.409+0000 7f8ebc2f7dc0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2020-05-18T21:40:02.889 INFO:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-05-18T21:40:02.947 INFO:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:40:02.981 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:02.977+0000 7f132d48adc0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2020-05-18T21:40:02.982 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:02.977+0000 7f132d48adc0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2020-05-18T21:40:02.982 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:02.977+0000 7f132d48adc0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-05-18T21:40:02.982 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:02.977+0000 7f132d48adc0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-05-18T21:40:02.982 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:02.977+0000 7f132d48adc0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-05-18T21:40:02.983 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:02.977+0000 7f132d48adc0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2020-05-18T21:40:05.456 INFO:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-05-18T21:40:05.472 INFO:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:40:05.553 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:05.549+0000 7fa49353bdc0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2020-05-18T21:40:05.554 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:05.549+0000 7fa49353bdc0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2020-05-18T21:40:05.554 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:05.549+0000 7fa49353bdc0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-05-18T21:40:05.554 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:05.549+0000 7fa49353bdc0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-05-18T21:40:05.555 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:05.549+0000 7fa49353bdc0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-05-18T21:40:05.555 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:05.553+0000 7fa49353bdc0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2020-05-18T21:40:08.025 INFO:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-05-18T21:40:08.043 INFO:tasks.ceph:Reading keys from all nodes... 2020-05-18T21:40:08.044 INFO:teuthology.orchestra.run.smithi196:> mktemp 2020-05-18T21:40:08.053 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.UHy7X9hITg 2020-05-18T21:40:08.054 INFO:teuthology.orchestra.run.smithi196:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.UHy7X9hITg 2020-05-18T21:40:08.114 INFO:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.UHy7X9hITg 2020-05-18T21:40:08.134 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.UHy7X9hITg is 56B 2020-05-18T21:40:08.148 INFO:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.UHy7X9hITg 2020-05-18T21:40:08.153 INFO:teuthology.orchestra.run.smithi196:> mktemp 2020-05-18T21:40:08.199 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.Jk3xcFXUXL 2020-05-18T21:40:08.200 INFO:teuthology.orchestra.run.smithi196:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.Jk3xcFXUXL 2020-05-18T21:40:08.215 INFO:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.Jk3xcFXUXL 2020-05-18T21:40:08.242 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.Jk3xcFXUXL is 56B 2020-05-18T21:40:08.254 INFO:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.Jk3xcFXUXL 2020-05-18T21:40:08.260 INFO:teuthology.orchestra.run.smithi196:> mktemp 2020-05-18T21:40:08.307 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.M7MGvspWoi 2020-05-18T21:40:08.308 INFO:teuthology.orchestra.run.smithi196:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.M7MGvspWoi 2020-05-18T21:40:08.321 INFO:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.M7MGvspWoi 2020-05-18T21:40:08.343 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.M7MGvspWoi is 56B 2020-05-18T21:40:08.355 INFO:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.M7MGvspWoi 2020-05-18T21:40:08.361 INFO:teuthology.orchestra.run.smithi196:> mktemp 2020-05-18T21:40:08.407 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.iXJUVpZaNe 2020-05-18T21:40:08.408 INFO:teuthology.orchestra.run.smithi196:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.iXJUVpZaNe 2020-05-18T21:40:08.422 INFO:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.iXJUVpZaNe 2020-05-18T21:40:08.448 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.iXJUVpZaNe is 56B 2020-05-18T21:40:08.498 INFO:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.iXJUVpZaNe 2020-05-18T21:40:08.505 INFO:teuthology.orchestra.run.smithi196:> mktemp 2020-05-18T21:40:08.511 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.fJSHh4Nwr4 2020-05-18T21:40:08.512 INFO:teuthology.orchestra.run.smithi196:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.fJSHh4Nwr4 2020-05-18T21:40:08.572 INFO:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.fJSHh4Nwr4 2020-05-18T21:40:08.598 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.fJSHh4Nwr4 is 56B 2020-05-18T21:40:08.611 INFO:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.fJSHh4Nwr4 2020-05-18T21:40:08.618 INFO:teuthology.orchestra.run.smithi196:> mktemp 2020-05-18T21:40:08.667 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.Jb8GlNDqP3 2020-05-18T21:40:08.668 INFO:teuthology.orchestra.run.smithi196:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.Jb8GlNDqP3 2020-05-18T21:40:08.685 INFO:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.Jb8GlNDqP3 2020-05-18T21:40:08.704 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.Jb8GlNDqP3 is 56B 2020-05-18T21:40:08.718 INFO:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.Jb8GlNDqP3 2020-05-18T21:40:08.725 INFO:teuthology.orchestra.run.smithi080:> mktemp 2020-05-18T21:40:08.731 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.P78cH9jYE9 2020-05-18T21:40:08.732 INFO:teuthology.orchestra.run.smithi080:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.P78cH9jYE9 2020-05-18T21:40:08.789 INFO:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.P78cH9jYE9 2020-05-18T21:40:08.811 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.P78cH9jYE9 is 56B 2020-05-18T21:40:08.823 INFO:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.P78cH9jYE9 2020-05-18T21:40:08.830 INFO:teuthology.orchestra.run.smithi080:> mktemp 2020-05-18T21:40:08.880 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.1YGHTzaMMq 2020-05-18T21:40:08.881 INFO:teuthology.orchestra.run.smithi080:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.1YGHTzaMMq 2020-05-18T21:40:08.896 INFO:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.1YGHTzaMMq 2020-05-18T21:40:08.924 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.1YGHTzaMMq is 56B 2020-05-18T21:40:08.945 INFO:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.1YGHTzaMMq 2020-05-18T21:40:08.953 INFO:teuthology.orchestra.run.smithi080:> mktemp 2020-05-18T21:40:09.000 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.opSWbEuYio 2020-05-18T21:40:09.000 INFO:teuthology.orchestra.run.smithi080:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.opSWbEuYio 2020-05-18T21:40:09.016 INFO:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.opSWbEuYio 2020-05-18T21:40:09.039 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.opSWbEuYio is 56B 2020-05-18T21:40:09.054 INFO:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.opSWbEuYio 2020-05-18T21:40:09.060 INFO:teuthology.orchestra.run.smithi080:> mktemp 2020-05-18T21:40:09.112 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.HLxNH7MFLB 2020-05-18T21:40:09.113 INFO:teuthology.orchestra.run.smithi080:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.HLxNH7MFLB 2020-05-18T21:40:09.126 INFO:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.HLxNH7MFLB 2020-05-18T21:40:09.189 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.HLxNH7MFLB is 56B 2020-05-18T21:40:09.203 INFO:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.HLxNH7MFLB 2020-05-18T21:40:09.209 INFO:teuthology.orchestra.run.smithi080:> mktemp 2020-05-18T21:40:09.257 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.hItr2gpwUo 2020-05-18T21:40:09.257 INFO:teuthology.orchestra.run.smithi080:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.hItr2gpwUo 2020-05-18T21:40:09.272 INFO:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.hItr2gpwUo 2020-05-18T21:40:09.290 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.hItr2gpwUo is 56B 2020-05-18T21:40:09.304 INFO:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.hItr2gpwUo 2020-05-18T21:40:09.310 INFO:teuthology.orchestra.run.smithi080:> mktemp 2020-05-18T21:40:09.356 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.AxFDrm1pdA 2020-05-18T21:40:09.356 INFO:teuthology.orchestra.run.smithi080:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.AxFDrm1pdA 2020-05-18T21:40:09.371 INFO:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.AxFDrm1pdA 2020-05-18T21:40:09.390 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.AxFDrm1pdA is 56B 2020-05-18T21:40:09.406 INFO:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.AxFDrm1pdA 2020-05-18T21:40:09.425 DEBUG:teuthology.orchestra.remote:smithi196:/etc/ceph/ceph.client.0.keyring is 59B 2020-05-18T21:40:09.438 INFO:tasks.ceph:Adding keys to all mons... 2020-05-18T21:40:09.439 INFO:teuthology.orchestra.run.smithi080:> sudo tee -a /etc/ceph/ceph.keyring 2020-05-18T21:40:09.456 INFO:teuthology.orchestra.run.smithi196:> sudo tee -a /etc/ceph/ceph.keyring 2020-05-18T21:40:09.466 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr.y] 2020-05-18T21:40:09.467 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCeAMNexZ4eEhAAzTZqGvtSFk0i1IQlqeJXkg== 2020-05-18T21:40:09.467 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.a] 2020-05-18T21:40:09.467 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCeAMNe5nhIGBAA8fqZL+BHSUW/vnrR7uL3qQ== 2020-05-18T21:40:09.467 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.0] 2020-05-18T21:40:09.468 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQChAMNeGqUDEBAA7s28A/TPeRi7XM9eRhCHsA== 2020-05-18T21:40:09.468 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.1] 2020-05-18T21:40:09.468 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCjAMNeS9FEMBAApJoLBgcqzmywomMFOS/BEQ== 2020-05-18T21:40:09.468 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.2] 2020-05-18T21:40:09.468 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCmAMNeHQ+6FRAATHsQuQMcD8IjGRMAGnws6w== 2020-05-18T21:40:09.469 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.3] 2020-05-18T21:40:09.469 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCoAMNew7WENBAAijhjkUjzH+CgTQKOdZwoyw== 2020-05-18T21:40:09.469 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr.x] 2020-05-18T21:40:09.469 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCeAMNenGZKFRAAe+/s6Gth3/XjLdz8Tnd+hg== 2020-05-18T21:40:09.469 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.b] 2020-05-18T21:40:09.470 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCeAMNeGYkcHhAAOF+QABC864VmZiIuBXwVmw== 2020-05-18T21:40:09.470 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.4] 2020-05-18T21:40:09.470 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCtAMNe2njyNBAA31AhnrYKIWwARcRuEukMqw== 2020-05-18T21:40:09.470 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.5] 2020-05-18T21:40:09.470 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCwAMNeK06yGBAApTs85EUnX7SC1Ku+LxOz0w== 2020-05-18T21:40:09.471 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.6] 2020-05-18T21:40:09.471 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCyAMNePiSKOhAAYeIdYAh4J0pe8YWNQwUsbQ== 2020-05-18T21:40:09.471 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.7] 2020-05-18T21:40:09.471 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQC1AMNej1EKIRAALqicNYOwIQxFaSfpitCiNA== 2020-05-18T21:40:09.471 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2020-05-18T21:40:09.471 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCeAMNeYWEhJBAAvEyOqWaHRUDZQ4P+xezO0A== 2020-05-18T21:40:09.473 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr.y] 2020-05-18T21:40:09.473 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCeAMNexZ4eEhAAzTZqGvtSFk0i1IQlqeJXkg== 2020-05-18T21:40:09.473 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2020-05-18T21:40:09.473 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCeAMNe5nhIGBAA8fqZL+BHSUW/vnrR7uL3qQ== 2020-05-18T21:40:09.474 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.0] 2020-05-18T21:40:09.474 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQChAMNeGqUDEBAA7s28A/TPeRi7XM9eRhCHsA== 2020-05-18T21:40:09.474 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.1] 2020-05-18T21:40:09.474 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCjAMNeS9FEMBAApJoLBgcqzmywomMFOS/BEQ== 2020-05-18T21:40:09.474 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.2] 2020-05-18T21:40:09.475 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCmAMNeHQ+6FRAATHsQuQMcD8IjGRMAGnws6w== 2020-05-18T21:40:09.475 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.3] 2020-05-18T21:40:09.475 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCoAMNew7WENBAAijhjkUjzH+CgTQKOdZwoyw== 2020-05-18T21:40:09.475 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr.x] 2020-05-18T21:40:09.475 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCeAMNenGZKFRAAe+/s6Gth3/XjLdz8Tnd+hg== 2020-05-18T21:40:09.476 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2020-05-18T21:40:09.476 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCeAMNeGYkcHhAAOF+QABC864VmZiIuBXwVmw== 2020-05-18T21:40:09.476 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.4] 2020-05-18T21:40:09.476 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCtAMNe2njyNBAA31AhnrYKIWwARcRuEukMqw== 2020-05-18T21:40:09.476 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.5] 2020-05-18T21:40:09.477 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCwAMNeK06yGBAApTs85EUnX7SC1Ku+LxOz0w== 2020-05-18T21:40:09.477 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.6] 2020-05-18T21:40:09.477 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCyAMNePiSKOhAAYeIdYAh4J0pe8YWNQwUsbQ== 2020-05-18T21:40:09.477 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.7] 2020-05-18T21:40:09.477 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC1AMNej1EKIRAALqicNYOwIQxFaSfpitCiNA== 2020-05-18T21:40:09.478 INFO:teuthology.orchestra.run.smithi196.stdout:[client.0] 2020-05-18T21:40:09.478 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQCeAMNeYWEhJBAAvEyOqWaHRUDZQ4P+xezO0A== 2020-05-18T21:40:09.478 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-05-18T21:40:09.481 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-05-18T21:40:09.555 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-05-18T21:40:09.564 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-05-18T21:40:09.610 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:09.612 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:09.694 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:09.697 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:09.738 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:09.780 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:09.824 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:09.826 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:09.906 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-05-18T21:40:09.909 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-05-18T21:40:09.951 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-05-18T21:40:09.954 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-05-18T21:40:09.996 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:09.999 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:10.080 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:10.087 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:10.132 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:10.168 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:10.221 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:10.224 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-18T21:40:10.265 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-05-18T21:40:10.268 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-05-18T21:40:10.311 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-05-18T21:40:10.311 INFO:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-05-18T21:40:10.372 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-05-18T21:40:10.444 INFO:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-05-18T21:40:10.461 INFO:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2020-05-18T21:40:10.478 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-05-18T21:40:10.536 INFO:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2020-05-18T21:40:10.552 INFO:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2020-05-18T21:40:10.611 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-05-18T21:40:10.674 INFO:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2020-05-18T21:40:10.690 INFO:teuthology.orchestra.run.smithi080:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:40:10.692 INFO:teuthology.orchestra.run.smithi196:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-05-18T21:40:10.699 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-05-18T21:40:10.699 INFO:tasks.ceph.mon.a:Restarting daemon 2020-05-18T21:40:10.700 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-05-18T21:40:10.702 INFO:tasks.ceph.mon.a:Started 2020-05-18T21:40:10.703 INFO:tasks.ceph.mon.b:Restarting daemon 2020-05-18T21:40:10.703 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-05-18T21:40:10.740 INFO:tasks.ceph.mon.b:Started 2020-05-18T21:40:10.741 INFO:tasks.ceph.mon.c:Restarting daemon 2020-05-18T21:40:10.741 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-05-18T21:40:10.743 INFO:tasks.ceph.mon.c:Started 2020-05-18T21:40:10.744 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-05-18T21:40:10.744 INFO:tasks.ceph.mgr.y:Restarting daemon 2020-05-18T21:40:10.745 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2020-05-18T21:40:10.747 INFO:tasks.ceph.mgr.y:Started 2020-05-18T21:40:10.748 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-05-18T21:40:10.748 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-05-18T21:40:10.750 INFO:tasks.ceph.mgr.x:Started 2020-05-18T21:40:10.751 INFO:tasks.ceph:Setting crush tunables to default 2020-05-18T21:40:10.751 INFO:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd crush tunables default 2020-05-18T21:40:11.356 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:40:11.359 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:40:15.956 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.843+0000 7f95cbfff700 1 Processor -- start 2020-05-18T21:40:15.957 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.843+0000 7f95cbfff700 1 -- start start 2020-05-18T21:40:15.958 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.843+0000 7f95cbfff700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 0x7f95c4036d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:15.958 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.843+0000 7f95cbfff700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 0x7f95c4038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:15.958 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.843+0000 7f95cbfff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 0x7f95c40ec460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:15.958 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.843+0000 7f95cbfff700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c4035a00 con 0x7f95c4038870 2020-05-18T21:40:15.958 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.843+0000 7f95cbfff700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c4036190 con 0x7f95c40ebfd0 2020-05-18T21:40:15.959 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.843+0000 7f95cbfff700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c4036430 con 0x7f95c40346c0 2020-05-18T21:40:15.959 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95cb7fe700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 0x7f95c4036d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.959 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95caffd700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 0x7f95c4038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95caffd700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 0x7f95c4038cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.196:37930/0 (socket says 172.21.15.196:37930) 2020-05-18T21:40:15.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95cb7fe700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 0x7f95c4036d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.196:37534/0 (socket says 172.21.15.196:37534) 2020-05-18T21:40:15.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95d1578700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 0x7f95c40ec460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95caffd700 1 -- 172.21.15.196:0/2842139038 learned_addr learned my addr 172.21.15.196:0/2842139038 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:15.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95caffd700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 msgr2=0x7f95c4038cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-05-18T21:40:15.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95caffd700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 msgr2=0x7f95c4038cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-05-18T21:40:15.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 0x7f95c4038cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted) 2020-05-18T21:40:15.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 0x7f95c4038cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.200000 2020-05-18T21:40:15.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95cb7fe700 1 -- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 msgr2=0x7f95c4036d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-05-18T21:40:15.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95cb7fe700 1 -- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 msgr2=0x7f95c4036d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-05-18T21:40:15.962 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 0x7f95c4036d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted) 2020-05-18T21:40:15.962 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95d1578700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 msgr2=0x7f95c40ec460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-05-18T21:40:15.962 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95d1578700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 msgr2=0x7f95c40ec460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-05-18T21:40:15.962 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 0x7f95c4036d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.200000 2020-05-18T21:40:15.962 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95d1578700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 0x7f95c40ec460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted) 2020-05-18T21:40:15.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:10.847+0000 7f95d1578700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 0x7f95c40ec460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.200000 2020-05-18T21:40:15.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95d1578700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 0x7f95c40ec460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 0x7f95c4036d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95d1578700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 msgr2=0x7f95c40ec460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-05-18T21:40:15.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95d1578700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 msgr2=0x7f95c40ec460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-05-18T21:40:15.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 0x7f95c4038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.964 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95d1578700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 0x7f95c40ec460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted) 2020-05-18T21:40:15.964 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95d1578700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 0x7f95c40ec460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.400000 2020-05-18T21:40:15.964 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95cb7fe700 1 -- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 msgr2=0x7f95c4036d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-05-18T21:40:15.964 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95cb7fe700 1 -- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 msgr2=0x7f95c4036d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-05-18T21:40:15.965 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 0x7f95c4036d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted) 2020-05-18T21:40:15.965 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95caffd700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 msgr2=0x7f95c4038cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-05-18T21:40:15.965 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 0x7f95c4036d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.400000 2020-05-18T21:40:15.965 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95caffd700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 msgr2=0x7f95c4038cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-05-18T21:40:15.965 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 0x7f95c4038cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted) 2020-05-18T21:40:15.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.051+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 0x7f95c4038cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.400000 2020-05-18T21:40:15.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95d1578700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 0x7f95c40ec460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95d1578700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 msgr2=0x7f95c40ec460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-05-18T21:40:15.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95d1578700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 msgr2=0x7f95c40ec460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-05-18T21:40:15.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 0x7f95c4036d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95d1578700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 0x7f95c40ec460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted) 2020-05-18T21:40:15.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 0x7f95c4038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95d1578700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 0x7f95c40ec460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.800000 2020-05-18T21:40:15.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95cb7fe700 1 -- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 msgr2=0x7f95c4036d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-05-18T21:40:15.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95cb7fe700 1 -- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 msgr2=0x7f95c4036d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-05-18T21:40:15.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 0x7f95c4036d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted) 2020-05-18T21:40:15.968 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 0x7f95c4036d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.800000 2020-05-18T21:40:15.968 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95caffd700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 msgr2=0x7f95c4038cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-05-18T21:40:15.968 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95caffd700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 msgr2=0x7f95c4038cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-05-18T21:40:15.969 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 0x7f95c4038cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted) 2020-05-18T21:40:15.969 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.455+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 0x7f95c4038cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.800000 2020-05-18T21:40:15.969 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.847+0000 7f95c97fa700 1 -- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 msgr2=0x7f95c4036d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-05-18T21:40:15.969 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.847+0000 7f95c97fa700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40346c0 0x7f95c4036d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:15.969 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.847+0000 7f95c97fa700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 msgr2=0x7f95c40ec460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-05-18T21:40:15.970 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.847+0000 7f95c97fa700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.196:3300/0 conn(0x7f95c40ebfd0 0x7f95c40ec460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:15.970 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.847+0000 7f95c97fa700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 msgr2=0x7f95c4038cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-05-18T21:40:15.970 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.847+0000 7f95c97fa700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95c4038870 0x7f95c4038cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:15.970 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.847+0000 7f95c97fa700 1 -- 172.21.15.196:0/2842139038 --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c4039110 con 0x7f95b8007020 2020-05-18T21:40:15.970 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:11.847+0000 7f95c97fa700 1 -- 172.21.15.196:0/2842139038 --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b800be20 con 0x7f95b8003370 2020-05-18T21:40:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:14.847+0000 7f95c97fa700 1 -- 172.21.15.196:0/2842139038 >> v1:172.21.15.196:6789/0 conn(0x7f95b8003370 legacy=0x7f95b80057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:14.847+0000 7f95c97fa700 1 -- 172.21.15.196:0/2842139038 >> v1:172.21.15.80:6789/0 conn(0x7f95b8007020 legacy=0x7f95b80094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:14.847+0000 7f95d1578700 1 -- 172.21.15.196:0/2842139038 reap_dead start 2020-05-18T21:40:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:14.847+0000 7f95c97fa700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 0x7f95b80057f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:14.847+0000 7f95c97fa700 1 -- 172.21.15.196:0/2842139038 --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b800d0e0 con 0x7f95b800b780 2020-05-18T21:40:15.971 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:14.847+0000 7f95c97fa700 1 -- 172.21.15.196:0/2842139038 --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b8006630 con 0x7f95b800b370 2020-05-18T21:40:15.972 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:14.847+0000 7f95c97fa700 1 -- 172.21.15.196:0/2842139038 --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c4035a00 con 0x7f95b800af90 2020-05-18T21:40:15.972 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:14.847+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 0x7f95b80057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.972 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:14.847+0000 7f95cb7fe700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 msgr2=0x7f95b80057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-05-18T21:40:15.972 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:14.847+0000 7f95cb7fe700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 msgr2=0x7f95b80057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-05-18T21:40:15.972 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:14.847+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 0x7f95b80057f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted) 2020-05-18T21:40:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:14.847+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 0x7f95b80057f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.200000 2020-05-18T21:40:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.051+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 0x7f95b80057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.051+0000 7f95cb7fe700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 msgr2=0x7f95b80057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-05-18T21:40:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.051+0000 7f95cb7fe700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 msgr2=0x7f95b80057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-05-18T21:40:15.973 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.051+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 0x7f95b80057f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted) 2020-05-18T21:40:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.051+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 0x7f95b80057f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.400000 2020-05-18T21:40:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.455+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 0x7f95b80057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.455+0000 7f95cb7fe700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 msgr2=0x7f95b80057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2020-05-18T21:40:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.455+0000 7f95cb7fe700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 msgr2=0x7f95b80057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-05-18T21:40:15.974 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.455+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 0x7f95b80057f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted) 2020-05-18T21:40:15.975 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.455+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 0x7f95b80057f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.800000 2020-05-18T21:40:15.975 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.939+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949058552 0 0) 0x7f95b800d0e0 con 0x7f95b800b780 2020-05-18T21:40:15.975 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.939+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95ac003270 con 0x7f95b800b780 2020-05-18T21:40:15.975 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.939+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038178276 0 0) 0x7f95b8006630 con 0x7f95b800b370 2020-05-18T21:40:15.975 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.939+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b800d0e0 con 0x7f95b800b370 2020-05-18T21:40:15.976 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 960615299 0 0) 0x7f95b800d0e0 con 0x7f95b800b370 2020-05-18T21:40:15.976 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 msgr2=0x7f95b80057f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-05-18T21:40:15.976 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95ca7fc700 1 --2- 172.21.15.196:0/2842139038 >> v2:172.21.15.80:3300/0 conn(0x7f95b800af90 0x7f95b80057f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:15.976 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 >> v1:172.21.15.80:6789/0 conn(0x7f95b800b780 legacy=0x7f95b8005f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:15.976 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b8006810 con 0x7f95b800b370 2020-05-18T21:40:15.977 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7f95b4008200 con 0x7f95b800b370 2020-05-18T21:40:15.977 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 >> v1:172.21.15.196:6789/0 conn(0x7f95b800b370 legacy=0x7f95b80094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:15.977 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95ca7fc700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95ac0038f0 0x7f95ac003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:15.977 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95ca7fc700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95ac004990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:15.977 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95ca7fc700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95ac004df0 0x7f95ac00b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:15.978 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95d1578700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95ac004df0 0x7f95ac00b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.978 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95ac004990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.978 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c4035a00 con 0x7f95ac0038f0 2020-05-18T21:40:15.978 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95ac0038f0 0x7f95ac003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.979 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac005330 con 0x7f95ac004df0 2020-05-18T21:40:15.979 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac00bbc0 con 0x7f95ac004510 2020-05-18T21:40:15.979 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95caffd700 1 -- 172.21.15.196:0/2842139038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95ac004df0 msgr2=0x7f95ac00b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:15.979 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95ac004df0 0x7f95ac00b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:15.979 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95caffd700 1 -- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95ac0038f0 msgr2=0x7f95ac003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:15.980 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95ac0038f0 0x7f95ac003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:15.980 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95caffd700 1 -- 172.21.15.196:0/2842139038 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b800d0e0 con 0x7f95ac004510 2020-05-18T21:40:15.980 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95d1578700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95ac004df0 0x7f95ac00b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:15.980 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95d1578700 1 -- 172.21.15.196:0/2842139038 reap_dead start 2020-05-18T21:40:15.980 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95ac004990 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f95ac004480 tx=0x7f95c403dbe0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:15.981 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95ac004990 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f95ac004480 tx=0x7f95c403dbe0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:15.981 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.943+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95ac004990 secure :-1 s=READY pgs=6 cs=0 l=1 rx=0x7f95ac004480 tx=0x7f95c403dbe0).ready entity=mon.2 client_cookie=544d4158e6c6cbcc server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:15.981 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.947+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95ac004990 secure :-1 s=THROTTLE_DONE pgs=6 cs=0 l=1 rx=0x7f95ac004480 tx=0x7f95c403dbe0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:15.981 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.947+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95ac004990 secure :-1 s=THROTTLE_DONE pgs=6 cs=0 l=1 rx=0x7f95ac004480 tx=0x7f95c403dbe0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:15.981 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.947+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95b4007180 con 0x7f95ac004510 2020-05-18T21:40:15.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.947+0000 7f95caffd700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95ac004990 secure :-1 s=THROTTLE_DONE pgs=6 cs=0 l=1 rx=0x7f95ac004480 tx=0x7f95c403dbe0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:15.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.947+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b40072e0 con 0x7f95ac004510 2020-05-18T21:40:15.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.947+0000 7f95ca7fc700 1 -- 172.21.15.196:0/2842139038 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95b40060d0 con 0x7f95ac004510 2020-05-18T21:40:15.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.947+0000 7f95cbfff700 1 -- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 msgr2=0x7f95ac004990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:15.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.947+0000 7f95cbfff700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95ac004990 secure :-1 s=READY pgs=6 cs=0 l=1 rx=0x7f95ac004480 tx=0x7f95c403dbe0).stop 2020-05-18T21:40:15.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.947+0000 7f95cbfff700 1 -- 172.21.15.196:0/2842139038 shutdown_connections 2020-05-18T21:40:15.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.947+0000 7f95cbfff700 1 --2- 172.21.15.196:0/2842139038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95ac004990 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:15.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.947+0000 7f95cbfff700 1 -- 172.21.15.196:0/2842139038 >> 172.21.15.196:0/2842139038 conn(0x7f95c403ceb0 msgr2=0x7f95c403d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:15.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.947+0000 7f95cbfff700 1 -- 172.21.15.196:0/2842139038 shutdown_connections 2020-05-18T21:40:15.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.947+0000 7f95cbfff700 1 -- 172.21.15.196:0/2842139038 wait complete. 2020-05-18T21:40:15.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.951+0000 7f95cbfff700 1 Processor -- start 2020-05-18T21:40:15.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.951+0000 7f95cbfff700 1 -- start start 2020-05-18T21:40:15.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.951+0000 7f95cbfff700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:15.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.951+0000 7f95cbfff700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c402ff90 con 0x7f95c40d1a40 2020-05-18T21:40:15.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.951+0000 7f95cbfff700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c40e89f0 con 0x7f95c40e27c0 2020-05-18T21:40:15.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.951+0000 7f95cbfff700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c4037550 con 0x7f95ac004510 2020-05-18T21:40:15.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.951+0000 7f95cb7fe700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:15.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.951+0000 7f95d1578700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f95c40e27c0 0x7f95c40b2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42544/0 (socket says 172.21.15.196:42544) 2020-05-18T21:40:15.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.951+0000 7f95d1578700 1 -- 172.21.15.196:0/947870571 learned_addr learned my addr 172.21.15.196:0/947870571 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:15.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.951+0000 7f95caffd700 1 --1- 172.21.15.196:0/947870571 >> v1:172.21.15.80:6789/0 conn(0x7f95c40d1a40 0x7f95c40e20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.196:38370/0 (socket says 172.21.15.196:38370) 2020-05-18T21:40:15.986 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.951+0000 7f95c3fff700 1 -- 172.21.15.196:0/947870571 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447753419 0 0) 0x7f95c402ff90 con 0x7f95c40d1a40 2020-05-18T21:40:15.986 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95c3fff700 1 -- 172.21.15.196:0/947870571 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b0002fe0 con 0x7f95c40d1a40 2020-05-18T21:40:15.986 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cb7fe700 1 -- 172.21.15.196:0/947870571 >> v1:172.21.15.196:6789/0 conn(0x7f95c40e27c0 legacy=0x7f95c40b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:15.986 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cb7fe700 1 -- 172.21.15.196:0/947870571 >> v1:172.21.15.80:6789/0 conn(0x7f95c40d1a40 legacy=0x7f95c40e20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:15.987 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cb7fe700 1 -- 172.21.15.196:0/947870571 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4037850 con 0x7f95ac004510 2020-05-18T21:40:15.987 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/947870571 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f95b800cfe0 tx=0x7f95c400aa50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:15.987 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/947870571 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f95b800cfe0 tx=0x7f95c400aa50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:15.988 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/947870571 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 secure :-1 s=READY pgs=7 cs=0 l=1 rx=0x7f95b800cfe0 tx=0x7f95c400aa50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:15.988 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/947870571 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 secure :-1 s=THROTTLE_DONE pgs=7 cs=0 l=1 rx=0x7f95b800cfe0 tx=0x7f95c400aa50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:15.988 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95c3fff700 1 -- 172.21.15.196:0/947870571 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95bc002080 con 0x7f95ac004510 2020-05-18T21:40:15.989 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/947870571 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 secure :-1 s=THROTTLE_DONE pgs=7 cs=0 l=1 rx=0x7f95b800cfe0 tx=0x7f95c400aa50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:15.989 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/947870571 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 secure :-1 s=THROTTLE_DONE pgs=7 cs=0 l=1 rx=0x7f95b800cfe0 tx=0x7f95c400aa50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:15.989 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95c3fff700 1 -- 172.21.15.196:0/947870571 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc0021e0 con 0x7f95ac004510 2020-05-18T21:40:15.989 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cbfff700 1 -- 172.21.15.196:0/947870571 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c40eb3f0 con 0x7f95ac004510 2020-05-18T21:40:15.989 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95c3fff700 1 -- 172.21.15.196:0/947870571 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95bc002540 con 0x7f95ac004510 2020-05-18T21:40:15.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cbfff700 1 -- 172.21.15.196:0/947870571 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c40e8c20 con 0x7f95ac004510 2020-05-18T21:40:15.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/947870571 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 secure :-1 s=THROTTLE_DONE pgs=7 cs=0 l=1 rx=0x7f95b800cfe0 tx=0x7f95c400aa50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:15.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/947870571 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 secure :-1 s=THROTTLE_DONE pgs=7 cs=0 l=1 rx=0x7f95b800cfe0 tx=0x7f95c400aa50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:15.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95c3fff700 1 -- 172.21.15.196:0/947870571 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 1) v1 ==== 514+0+0 (secure 0 0 0) 0x7f95bc002c40 con 0x7f95ac004510 2020-05-18T21:40:15.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95c3fff700 1 -- 172.21.15.196:0/947870571 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f95bc0120d0 con 0x7f95ac004510 2020-05-18T21:40:15.991 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.955+0000 7f95cbfff700 1 -- 172.21.15.196:0/947870571 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c40e9060 con 0x7f95ac004510 2020-05-18T21:40:15.991 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.963+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/947870571 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 secure :-1 s=THROTTLE_DONE pgs=7 cs=0 l=1 rx=0x7f95b800cfe0 tx=0x7f95c400aa50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:15.991 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:15.963+0000 7f95c3fff700 1 -- 172.21.15.196:0/947870571 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67708 (secure 0 0 0) 0x7f95bc024890 con 0x7f95ac004510 2020-05-18T21:40:16.045 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:16.043+0000 7f95cbfff700 1 -- 172.21.15.196:0/947870571 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f95c4004c70 con 0x7f95ac004510 2020-05-18T21:40:16.056 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:16.055+0000 7f95cb7fe700 1 --2- 172.21.15.196:0/947870571 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 secure :-1 s=THROTTLE_DONE pgs=7 cs=0 l=1 rx=0x7f95b800cfe0 tx=0x7f95c400aa50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:16.056 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:16.055+0000 7f95c3fff700 1 -- 172.21.15.196:0/947870571 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f95bc00a1a0 con 0x7f95ac004510 2020-05-18T21:40:16.057 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:16.055+0000 7f95cbfff700 1 -- 172.21.15.196:0/947870571 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 msgr2=0x7f95c4005970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:16.058 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:16.055+0000 7f95cbfff700 1 --2- 172.21.15.196:0/947870571 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 secure :-1 s=READY pgs=7 cs=0 l=1 rx=0x7f95b800cfe0 tx=0x7f95c400aa50).stop 2020-05-18T21:40:16.058 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:16.055+0000 7f95cbfff700 1 -- 172.21.15.196:0/947870571 shutdown_connections 2020-05-18T21:40:16.058 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:16.055+0000 7f95cbfff700 1 --2- 172.21.15.196:0/947870571 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95ac004510 0x7f95c4005970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:16.059 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:16.055+0000 7f95cbfff700 1 -- 172.21.15.196:0/947870571 >> 172.21.15.196:0/947870571 conn(0x7f95c403ceb0 msgr2=0x7f95c40bdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:16.059 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:16.059+0000 7f95cbfff700 1 -- 172.21.15.196:0/947870571 shutdown_connections 2020-05-18T21:40:16.059 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:16.059+0000 7f95cbfff700 1 -- 172.21.15.196:0/947870571 wait complete. 2020-05-18T21:40:16.059 INFO:teuthology.orchestra.run.smithi196.stderr:adjusted tunables profile to default 2020-05-18T21:40:16.069 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-05-18T21:40:16.070 INFO:teuthology.orchestra.run.smithi196:> mktemp 2020-05-18T21:40:16.075 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.7PuIfUpAV8 2020-05-18T21:40:16.075 INFO:teuthology.orchestra.run.smithi196:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.7PuIfUpAV8 2020-05-18T21:40:16.134 INFO:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.7PuIfUpAV8 2020-05-18T21:40:16.151 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.7PuIfUpAV8 is 37B 2020-05-18T21:40:16.202 INFO:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.7PuIfUpAV8 2020-05-18T21:40:16.209 INFO:teuthology.orchestra.run.smithi196:> mktemp 2020-05-18T21:40:16.216 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.YXmHaP6lTr 2020-05-18T21:40:16.216 INFO:teuthology.orchestra.run.smithi196:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.YXmHaP6lTr 2020-05-18T21:40:16.229 INFO:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.YXmHaP6lTr 2020-05-18T21:40:16.292 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.YXmHaP6lTr is 37B 2020-05-18T21:40:16.302 INFO:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.YXmHaP6lTr 2020-05-18T21:40:16.308 INFO:teuthology.orchestra.run.smithi196:> mktemp 2020-05-18T21:40:16.355 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.xPt9KHOq9Y 2020-05-18T21:40:16.355 INFO:teuthology.orchestra.run.smithi196:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.xPt9KHOq9Y 2020-05-18T21:40:16.370 INFO:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.xPt9KHOq9Y 2020-05-18T21:40:16.392 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.xPt9KHOq9Y is 37B 2020-05-18T21:40:16.402 INFO:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.xPt9KHOq9Y 2020-05-18T21:40:16.407 INFO:teuthology.orchestra.run.smithi196:> mktemp 2020-05-18T21:40:16.455 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.pFrDxZXmDb 2020-05-18T21:40:16.455 INFO:teuthology.orchestra.run.smithi196:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.pFrDxZXmDb 2020-05-18T21:40:16.470 INFO:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.pFrDxZXmDb 2020-05-18T21:40:16.498 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.pFrDxZXmDb is 37B 2020-05-18T21:40:16.518 INFO:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.pFrDxZXmDb 2020-05-18T21:40:16.525 INFO:teuthology.orchestra.run.smithi080:> mktemp 2020-05-18T21:40:16.532 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.xWEAlXbnl4 2020-05-18T21:40:16.532 INFO:teuthology.orchestra.run.smithi080:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.xWEAlXbnl4 2020-05-18T21:40:16.545 INFO:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.xWEAlXbnl4 2020-05-18T21:40:16.570 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.xWEAlXbnl4 is 37B 2020-05-18T21:40:16.583 INFO:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.xWEAlXbnl4 2020-05-18T21:40:16.589 INFO:teuthology.orchestra.run.smithi080:> mktemp 2020-05-18T21:40:16.636 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.BzPqn5FRNk 2020-05-18T21:40:16.637 INFO:teuthology.orchestra.run.smithi080:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.BzPqn5FRNk 2020-05-18T21:40:16.651 INFO:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.BzPqn5FRNk 2020-05-18T21:40:16.712 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.BzPqn5FRNk is 37B 2020-05-18T21:40:16.727 INFO:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.BzPqn5FRNk 2020-05-18T21:40:16.734 INFO:teuthology.orchestra.run.smithi080:> mktemp 2020-05-18T21:40:16.780 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.rIrMtOsPwv 2020-05-18T21:40:16.781 INFO:teuthology.orchestra.run.smithi080:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.rIrMtOsPwv 2020-05-18T21:40:16.798 INFO:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.rIrMtOsPwv 2020-05-18T21:40:16.861 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.rIrMtOsPwv is 37B 2020-05-18T21:40:16.875 INFO:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.rIrMtOsPwv 2020-05-18T21:40:16.881 INFO:teuthology.orchestra.run.smithi080:> mktemp 2020-05-18T21:40:16.932 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.m4N6CYrxEp 2020-05-18T21:40:16.933 INFO:teuthology.orchestra.run.smithi080:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.m4N6CYrxEp 2020-05-18T21:40:16.947 INFO:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.m4N6CYrxEp 2020-05-18T21:40:16.967 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.m4N6CYrxEp is 37B 2020-05-18T21:40:16.978 INFO:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.m4N6CYrxEp 2020-05-18T21:40:16.995 INFO:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 8db63a0a-61dc-42b5-9559-fddcf436cab5 0 2020-05-18T21:40:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.133+0000 7f6967fff700 1 Processor -- start 2020-05-18T21:40:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.133+0000 7f6967fff700 1 -- start start 2020-05-18T21:40:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6967fff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6960038870 0x7f6960038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6967fff700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69600ebfd0 0x7f69600ec460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:17.143 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6967fff700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6960039140 con 0x7f69600346c0 2020-05-18T21:40:17.144 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6967fff700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6960035a00 con 0x7f6960038870 2020-05-18T21:40:17.144 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6967fff700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6960036190 con 0x7f69600ebfd0 2020-05-18T21:40:17.144 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f696d78b700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69600ebfd0 0x7f69600ec460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.144 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f696d78b700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69600ebfd0 0x7f69600ec460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:37458/0 (socket says 172.21.15.80:37458) 2020-05-18T21:40:17.145 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f696d78b700 1 -- 172.21.15.80:0/380311808 learned_addr learned my addr 172.21.15.80:0/380311808 (peer_addr_for_me v2:172.21.15.80:0/0) 2020-05-18T21:40:17.145 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6966ffd700 1 --2- 172.21.15.80:0/380311808 >> v2:172.21.15.196:3300/0 conn(0x7f6960038870 0x7f6960038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.145 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f696d78b700 1 -- 172.21.15.80:0/380311808 >> v2:172.21.15.196:3300/0 conn(0x7f6960038870 msgr2=0x7f6960038ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:17.145 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f696d78b700 1 --2- 172.21.15.80:0/380311808 >> v2:172.21.15.196:3300/0 conn(0x7f6960038870 0x7f6960038ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:17.146 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f696d78b700 1 -- 172.21.15.80:0/380311808 >> v1:172.21.15.196:6789/0 conn(0x7f69600346c0 legacy=0x7f6960036d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.146 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f696d78b700 1 -- 172.21.15.80:0/380311808 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960036430 con 0x7f69600ebfd0 2020-05-18T21:40:17.146 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6966ffd700 1 --2- 172.21.15.80:0/380311808 >> v2:172.21.15.196:3300/0 conn(0x7f6960038870 0x7f6960038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_reply_more state changed! 2020-05-18T21:40:17.146 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f696d78b700 1 --2- 172.21.15.80:0/380311808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69600ebfd0 0x7f69600ec460 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f6960035890 tx=0x7f6958008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.146 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f696d78b700 1 --2- 172.21.15.80:0/380311808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69600ebfd0 0x7f69600ec460 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f6960035890 tx=0x7f6958008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.147 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f696d78b700 1 --2- 172.21.15.80:0/380311808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69600ebfd0 0x7f69600ec460 secure :-1 s=READY pgs=9 cs=0 l=1 rx=0x7f6960035890 tx=0x7f6958008920).ready entity=mon.2 client_cookie=409370509d18f19b server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:17.147 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f696d78b700 1 --2- 172.21.15.80:0/380311808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69600ebfd0 0x7f69600ec460 secure :-1 s=THROTTLE_DONE pgs=9 cs=0 l=1 rx=0x7f6960035890 tx=0x7f6958008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.147 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f696d78b700 1 --2- 172.21.15.80:0/380311808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69600ebfd0 0x7f69600ec460 secure :-1 s=THROTTLE_DONE pgs=9 cs=0 l=1 rx=0x7f6960035890 tx=0x7f6958008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.147 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f696d78b700 1 --2- 172.21.15.80:0/380311808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69600ebfd0 0x7f69600ec460 secure :-1 s=THROTTLE_DONE pgs=9 cs=0 l=1 rx=0x7f6960035890 tx=0x7f6958008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.147 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f69667fc700 1 -- 172.21.15.80:0/380311808 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6958011760 con 0x7f69600ebfd0 2020-05-18T21:40:17.148 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f69667fc700 1 -- 172.21.15.80:0/380311808 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69580118c0 con 0x7f69600ebfd0 2020-05-18T21:40:17.148 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f69667fc700 1 -- 172.21.15.80:0/380311808 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6958011ce0 con 0x7f69600ebfd0 2020-05-18T21:40:17.148 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6967fff700 1 -- 172.21.15.80:0/380311808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69600ebfd0 msgr2=0x7f69600ec460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.148 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6967fff700 1 --2- 172.21.15.80:0/380311808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69600ebfd0 0x7f69600ec460 secure :-1 s=READY pgs=9 cs=0 l=1 rx=0x7f6960035890 tx=0x7f6958008920).stop 2020-05-18T21:40:17.148 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6967fff700 1 -- 172.21.15.80:0/380311808 shutdown_connections 2020-05-18T21:40:17.148 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6967fff700 1 --2- 172.21.15.80:0/380311808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69600ebfd0 0x7f69600ec460 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.149 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6967fff700 1 --2- 172.21.15.80:0/380311808 >> v2:172.21.15.196:3300/0 conn(0x7f6960038870 0x7f6960038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:17.149 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6967fff700 1 -- 172.21.15.80:0/380311808 >> 172.21.15.80:0/380311808 conn(0x7f696003ceb0 msgr2=0x7f696003d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:17.149 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6967fff700 1 -- 172.21.15.80:0/380311808 shutdown_connections 2020-05-18T21:40:17.150 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.137+0000 7f6967fff700 1 -- 172.21.15.80:0/380311808 wait complete. 2020-05-18T21:40:17.153 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f6967fff700 1 Processor -- start 2020-05-18T21:40:17.153 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f6967fff700 1 -- start start 2020-05-18T21:40:17.153 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f6967fff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f69600ebfd0 0x7f69600c3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:17.153 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f6967fff700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6960032480 con 0x7f6960038870 2020-05-18T21:40:17.154 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f6967fff700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6960030cc0 con 0x7f69600346c0 2020-05-18T21:40:17.154 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f6967fff700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6960030f60 con 0x7f69600ebfd0 2020-05-18T21:40:17.154 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f6966ffd700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f6960038870 0x7f6960031400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35310/0 (socket says 172.21.15.80:35310) 2020-05-18T21:40:17.154 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f6966ffd700 1 -- 172.21.15.80:0/2996913133 learned_addr learned my addr 172.21.15.80:0/2996913133 (peer_addr_for_me v1:172.21.15.80:0/0) 2020-05-18T21:40:17.155 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f696d78b700 1 --2- 172.21.15.80:0/2996913133 >> v2:172.21.15.196:3300/0 conn(0x7f69600ebfd0 0x7f69600c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.155 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f694ffff700 1 -- 172.21.15.80:0/2996913133 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073021220 0 0) 0x7f6960032480 con 0x7f6960038870 2020-05-18T21:40:17.155 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f694ffff700 1 -- 172.21.15.80:0/2996913133 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6948003270 con 0x7f6960038870 2020-05-18T21:40:17.155 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f696d78b700 1 -- 172.21.15.80:0/2996913133 >> v1:172.21.15.196:6789/0 conn(0x7f69600346c0 legacy=0x7f6960031cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.155 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f696d78b700 1 -- 172.21.15.80:0/2996913133 >> v1:172.21.15.80:6789/0 conn(0x7f6960038870 legacy=0x7f6960031400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.156 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f694ffff700 1 -- 172.21.15.80:0/2996913133 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 210521051 0 0) 0x7f6948003270 con 0x7f6960038870 2020-05-18T21:40:17.156 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f696d78b700 1 -- 172.21.15.80:0/2996913133 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960029940 con 0x7f69600ebfd0 2020-05-18T21:40:17.156 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f696d78b700 1 --2- 172.21.15.80:0/2996913133 >> v2:172.21.15.196:3300/0 conn(0x7f69600ebfd0 0x7f69600c3740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f695c0026e0 tx=0x7f6958011510).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.156 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f696d78b700 1 --2- 172.21.15.80:0/2996913133 >> v2:172.21.15.196:3300/0 conn(0x7f69600ebfd0 0x7f69600c3740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f695c0026e0 tx=0x7f6958011510).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.156 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f696d78b700 1 --2- 172.21.15.80:0/2996913133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69600ebfd0 0x7f69600c3740 secure :-1 s=READY pgs=18 cs=0 l=1 rx=0x7f695c0026e0 tx=0x7f6958011510).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:17.157 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f696d78b700 1 --2- 172.21.15.80:0/2996913133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69600ebfd0 0x7f69600c3740 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7f695c0026e0 tx=0x7f6958011510).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.157 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f694ffff700 1 -- 172.21.15.80:0/2996913133 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f695c002540 con 0x7f69600ebfd0 2020-05-18T21:40:17.157 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f696d78b700 1 --2- 172.21.15.80:0/2996913133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69600ebfd0 0x7f69600c3740 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7f695c0026e0 tx=0x7f6958011510).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.157 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f696d78b700 1 --2- 172.21.15.80:0/2996913133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69600ebfd0 0x7f69600c3740 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7f695c0026e0 tx=0x7f6958011510).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.157 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f694ffff700 1 -- 172.21.15.80:0/2996913133 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69580112e0 con 0x7f69600ebfd0 2020-05-18T21:40:17.158 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f694ffff700 1 -- 172.21.15.80:0/2996913133 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6958011c40 con 0x7f69600ebfd0 2020-05-18T21:40:17.158 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f6967fff700 1 -- 172.21.15.80:0/2996913133 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69600cb720 con 0x7f69600ebfd0 2020-05-18T21:40:17.158 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f6967fff700 1 -- 172.21.15.80:0/2996913133 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69600cbee0 con 0x7f69600ebfd0 2020-05-18T21:40:17.158 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f696d78b700 1 --2- 172.21.15.80:0/2996913133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69600ebfd0 0x7f69600c3740 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7f695c0026e0 tx=0x7f6958011510).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.158 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f694ffff700 1 -- 172.21.15.80:0/2996913133 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) v1 ==== 514+0+0 (secure 0 0 0) 0x7f6958005c00 con 0x7f69600ebfd0 2020-05-18T21:40:17.159 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f696d78b700 1 --2- 172.21.15.80:0/2996913133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69600ebfd0 0x7f69600c3740 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7f695c0026e0 tx=0x7f6958011510).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.159 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.141+0000 7f694ffff700 1 -- 172.21.15.80:0/2996913133 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f6958010e00 con 0x7f69600ebfd0 2020-05-18T21:40:17.159 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.145+0000 7f6967fff700 1 -- 172.21.15.80:0/2996913133 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6960004c70 con 0x7f69600ebfd0 2020-05-18T21:40:17.160 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.149+0000 7f696d78b700 1 --2- 172.21.15.80:0/2996913133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69600ebfd0 0x7f69600c3740 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7f695c0026e0 tx=0x7f6958011510).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.160 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.149+0000 7f694ffff700 1 -- 172.21.15.80:0/2996913133 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67708 (secure 0 0 0) 0x7f6958007180 con 0x7f69600ebfd0 2020-05-18T21:40:17.223 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.221+0000 7f6967fff700 1 -- 172.21.15.80:0/2996913133 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8db63a0a-61dc-42b5-9559-fddcf436cab5", "id": 0} v 0) v1 -- 0x7f69600e5f10 con 0x7f69600ebfd0 2020-05-18T21:40:17.233 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.229+0000 7f696d78b700 1 --2- 172.21.15.80:0/2996913133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69600ebfd0 0x7f69600c3740 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7f695c0026e0 tx=0x7f6958011510).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.233 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.229+0000 7f694ffff700 1 -- 172.21.15.80:0/2996913133 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8db63a0a-61dc-42b5-9559-fddcf436cab5", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6958007320 con 0x7f69600ebfd0 2020-05-18T21:40:17.234 INFO:teuthology.orchestra.run.smithi080.stdout:0 2020-05-18T21:40:17.235 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.229+0000 7f6967fff700 1 -- 172.21.15.80:0/2996913133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69600ebfd0 msgr2=0x7f69600c3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.235 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.229+0000 7f6967fff700 1 --2- 172.21.15.80:0/2996913133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69600ebfd0 0x7f69600c3740 secure :-1 s=READY pgs=18 cs=0 l=1 rx=0x7f695c0026e0 tx=0x7f6958011510).stop 2020-05-18T21:40:17.235 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.229+0000 7f6967fff700 1 -- 172.21.15.80:0/2996913133 shutdown_connections 2020-05-18T21:40:17.235 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.229+0000 7f6967fff700 1 --2- 172.21.15.80:0/2996913133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69600ebfd0 0x7f69600c3740 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.236 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.229+0000 7f6967fff700 1 -- 172.21.15.80:0/2996913133 >> 172.21.15.80:0/2996913133 conn(0x7f696003ceb0 msgr2=0x7f696003d290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:17.236 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.229+0000 7f6967fff700 1 -- 172.21.15.80:0/2996913133 shutdown_connections 2020-05-18T21:40:17.236 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.229+0000 7f6967fff700 1 -- 172.21.15.80:0/2996913133 wait complete. 2020-05-18T21:40:17.246 INFO:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 6773b0c3-9f49-42a2-b75b-c10c734ff086 1 2020-05-18T21:40:17.349 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7bb251700 1 Processor -- start 2020-05-18T21:40:17.350 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7bb251700 1 -- start start 2020-05-18T21:40:17.350 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7bb251700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7b40a7070 0x7fa7b40a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:17.350 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7bb251700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa7b40a4d70 0x7fa7b40a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:17.351 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7bb251700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b40a3eb0 con 0x7fa7b40a6720 2020-05-18T21:40:17.351 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7bb251700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b40a5f00 con 0x7fa7b40a4d70 2020-05-18T21:40:17.351 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7bb251700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b40a4150 con 0x7fa7b40a7070 2020-05-18T21:40:17.351 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b97ee700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7b40a7070 0x7fa7b40a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.351 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b97ee700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7b40a7070 0x7fa7b40a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:37468/0 (socket says 172.21.15.80:37468) 2020-05-18T21:40:17.352 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b9fef700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa7b40a6720 0x7fa7b40a1810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35318/0 (socket says 172.21.15.80:35318) 2020-05-18T21:40:17.352 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b97ee700 1 -- 172.21.15.80:0/300202762 learned_addr learned my addr 172.21.15.80:0/300202762 (peer_addr_for_me v2:172.21.15.80:0/0) 2020-05-18T21:40:17.352 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b8fed700 1 --2- 172.21.15.80:0/300202762 >> v2:172.21.15.196:3300/0 conn(0x7fa7b40a4d70 0x7fa7b40a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.352 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b97ee700 1 -- 172.21.15.80:0/300202762 >> v2:172.21.15.196:3300/0 conn(0x7fa7b40a4d70 msgr2=0x7fa7b40a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:17.352 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b97ee700 1 --2- 172.21.15.80:0/300202762 >> v2:172.21.15.196:3300/0 conn(0x7fa7b40a4d70 0x7fa7b40a62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:17.353 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b97ee700 1 -- 172.21.15.80:0/300202762 >> v1:172.21.15.80:6789/0 conn(0x7fa7b40a6720 legacy=0x7fa7b40a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.353 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b97ee700 1 -- 172.21.15.80:0/300202762 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b40a3be0 con 0x7fa7b40a7070 2020-05-18T21:40:17.353 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7abfff700 1 -- 172.21.15.80:0/300202762 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906956781 0 0) 0x7fa7b40a3eb0 con 0x7fa7b40a6720 2020-05-18T21:40:17.353 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b97ee700 1 --2- 172.21.15.80:0/300202762 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7b40a7070 0x7fa7b40a45f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fa7a4002f70 tx=0x7fa7a4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.353 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b97ee700 1 --2- 172.21.15.80:0/300202762 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7b40a7070 0x7fa7b40a45f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fa7a4002f70 tx=0x7fa7a4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.354 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b97ee700 1 --2- 172.21.15.80:0/300202762 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7b40a7070 0x7fa7b40a45f0 secure :-1 s=READY pgs=10 cs=0 l=1 rx=0x7fa7a4002f70 tx=0x7fa7a4007a80).ready entity=mon.2 client_cookie=3f88f125f0fceb74 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:17.354 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b97ee700 1 --2- 172.21.15.80:0/300202762 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7b40a7070 0x7fa7b40a45f0 secure :-1 s=THROTTLE_DONE pgs=10 cs=0 l=1 rx=0x7fa7a4002f70 tx=0x7fa7a4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.354 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b97ee700 1 --2- 172.21.15.80:0/300202762 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7b40a7070 0x7fa7b40a45f0 secure :-1 s=THROTTLE_DONE pgs=10 cs=0 l=1 rx=0x7fa7a4002f70 tx=0x7fa7a4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.354 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7abfff700 1 -- 172.21.15.80:0/300202762 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7a40116b0 con 0x7fa7b40a7070 2020-05-18T21:40:17.355 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7b97ee700 1 --2- 172.21.15.80:0/300202762 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7b40a7070 0x7fa7b40a45f0 secure :-1 s=THROTTLE_DONE pgs=10 cs=0 l=1 rx=0x7fa7a4002f70 tx=0x7fa7a4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.355 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7abfff700 1 -- 172.21.15.80:0/300202762 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a4011810 con 0x7fa7b40a7070 2020-05-18T21:40:17.355 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7abfff700 1 -- 172.21.15.80:0/300202762 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7a4011bf0 con 0x7fa7b40a7070 2020-05-18T21:40:17.355 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7bb251700 1 -- 172.21.15.80:0/300202762 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7b40a7070 msgr2=0x7fa7b40a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.355 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.341+0000 7fa7bb251700 1 --2- 172.21.15.80:0/300202762 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7b40a7070 0x7fa7b40a45f0 secure :-1 s=READY pgs=10 cs=0 l=1 rx=0x7fa7a4002f70 tx=0x7fa7a4007a80).stop 2020-05-18T21:40:17.355 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7bb251700 1 -- 172.21.15.80:0/300202762 shutdown_connections 2020-05-18T21:40:17.356 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7bb251700 1 --2- 172.21.15.80:0/300202762 >> v2:172.21.15.196:3300/0 conn(0x7fa7b40a4d70 0x7fa7b40a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:17.356 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7bb251700 1 --2- 172.21.15.80:0/300202762 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7b40a7070 0x7fa7b40a45f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.356 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7bb251700 1 -- 172.21.15.80:0/300202762 >> 172.21.15.80:0/300202762 conn(0x7fa7b405a7c0 msgr2=0x7fa7b405aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:17.356 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7bb251700 1 -- 172.21.15.80:0/300202762 shutdown_connections 2020-05-18T21:40:17.357 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7bb251700 1 -- 172.21.15.80:0/300202762 wait complete. 2020-05-18T21:40:17.357 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7bb251700 1 Processor -- start 2020-05-18T21:40:17.357 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7bb251700 1 -- start start 2020-05-18T21:40:17.360 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7bb251700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa7b40a4d70 0x7fa7b409c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:17.361 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7bb251700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa7b40a7070 0x7fa7b407e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:17.361 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7bb251700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b4065760 con 0x7fa7b40a6720 2020-05-18T21:40:17.361 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7bb251700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b409c6d0 con 0x7fa7b40a7070 2020-05-18T21:40:17.361 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7bb251700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b407f690 con 0x7fa7b40a4d70 2020-05-18T21:40:17.361 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7b9fef700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa7b40a7070 0x7fa7b407e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7b9fef700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa7b40a7070 0x7fa7b407e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33212/0 (socket says 172.21.15.80:33212) 2020-05-18T21:40:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7b8fed700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa7b40a6720 0x7fa7b4066140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35322/0 (socket says 172.21.15.80:35322) 2020-05-18T21:40:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7b9fef700 1 -- 172.21.15.80:0/3525042227 learned_addr learned my addr 172.21.15.80:0/3525042227 (peer_addr_for_me v2:172.21.15.80:0/0) 2020-05-18T21:40:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.345+0000 7fa7b97ee700 1 --2- 172.21.15.80:0/3525042227 >> v2:172.21.15.196:3300/0 conn(0x7fa7b40a4d70 0x7fa7b409c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.363 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7b9fef700 1 -- 172.21.15.80:0/3525042227 >> v2:172.21.15.196:3300/0 conn(0x7fa7b40a4d70 msgr2=0x7fa7b409c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.363 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7b9fef700 1 --2- 172.21.15.80:0/3525042227 >> v2:172.21.15.196:3300/0 conn(0x7fa7b40a4d70 0x7fa7b409c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.363 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7b9fef700 1 -- 172.21.15.80:0/3525042227 >> v1:172.21.15.80:6789/0 conn(0x7fa7b40a6720 legacy=0x7fa7b4066140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.363 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7a9ffb700 1 -- 172.21.15.80:0/3525042227 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761241495 0 0) 0x7fa7b4065760 con 0x7fa7b40a6720 2020-05-18T21:40:17.364 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7b9fef700 1 -- 172.21.15.80:0/3525042227 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b4125810 con 0x7fa7b40a7070 2020-05-18T21:40:17.364 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7b9fef700 1 --2- 172.21.15.80:0/3525042227 >> v2:172.21.15.80:3300/0 conn(0x7fa7b40a7070 0x7fa7b407e790 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa7ac000bc0 tx=0x7fa7ac010920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.364 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7b9fef700 1 --2- 172.21.15.80:0/3525042227 >> v2:172.21.15.80:3300/0 conn(0x7fa7b40a7070 0x7fa7b407e790 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa7ac000bc0 tx=0x7fa7ac010920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.364 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7b9fef700 1 --2- 172.21.15.80:0/3525042227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7b40a7070 0x7fa7b407e790 secure :-1 s=READY pgs=14 cs=0 l=1 rx=0x7fa7ac000bc0 tx=0x7fa7ac010920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:17.365 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7b9fef700 1 --2- 172.21.15.80:0/3525042227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7b40a7070 0x7fa7b407e790 secure :-1 s=THROTTLE_DONE pgs=14 cs=0 l=1 rx=0x7fa7ac000bc0 tx=0x7fa7ac010920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.365 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7a9ffb700 1 -- 172.21.15.80:0/3525042227 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7ac011630 con 0x7fa7b40a7070 2020-05-18T21:40:17.365 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7b9fef700 1 --2- 172.21.15.80:0/3525042227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7b40a7070 0x7fa7b407e790 secure :-1 s=THROTTLE_DONE pgs=14 cs=0 l=1 rx=0x7fa7ac000bc0 tx=0x7fa7ac010920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.365 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7b9fef700 1 --2- 172.21.15.80:0/3525042227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7b40a7070 0x7fa7b407e790 secure :-1 s=THROTTLE_DONE pgs=14 cs=0 l=1 rx=0x7fa7ac000bc0 tx=0x7fa7ac010920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.366 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7a9ffb700 1 -- 172.21.15.80:0/3525042227 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac011790 con 0x7fa7b40a7070 2020-05-18T21:40:17.366 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7a9ffb700 1 -- 172.21.15.80:0/3525042227 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7ac011b30 con 0x7fa7b40a7070 2020-05-18T21:40:17.366 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7bb251700 1 -- 172.21.15.80:0/3525042227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b41255d0 con 0x7fa7b40a7070 2020-05-18T21:40:17.366 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7bb251700 1 -- 172.21.15.80:0/3525042227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b41251f0 con 0x7fa7b40a7070 2020-05-18T21:40:17.367 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7b9fef700 1 --2- 172.21.15.80:0/3525042227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7b40a7070 0x7fa7b407e790 secure :-1 s=THROTTLE_DONE pgs=14 cs=0 l=1 rx=0x7fa7ac000bc0 tx=0x7fa7ac010920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.367 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7a9ffb700 1 -- 172.21.15.80:0/3525042227 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) v1 ==== 514+0+0 (secure 0 0 0) 0x7fa7ac00b870 con 0x7fa7b40a7070 2020-05-18T21:40:17.367 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7b9fef700 1 --2- 172.21.15.80:0/3525042227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7b40a7070 0x7fa7b407e790 secure :-1 s=THROTTLE_DONE pgs=14 cs=0 l=1 rx=0x7fa7ac000bc0 tx=0x7fa7ac010920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.367 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7a9ffb700 1 -- 172.21.15.80:0/3525042227 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fa7ac00abd0 con 0x7fa7b40a7070 2020-05-18T21:40:17.368 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.349+0000 7fa7bb251700 1 -- 172.21.15.80:0/3525042227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b4065e20 con 0x7fa7b40a7070 2020-05-18T21:40:17.368 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.353+0000 7fa7b9fef700 1 --2- 172.21.15.80:0/3525042227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7b40a7070 0x7fa7b407e790 secure :-1 s=THROTTLE_DONE pgs=14 cs=0 l=1 rx=0x7fa7ac000bc0 tx=0x7fa7ac010920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.368 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.353+0000 7fa7a9ffb700 1 -- 172.21.15.80:0/3525042227 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67708 (secure 0 0 0) 0x7fa7b41255d0 con 0x7fa7b40a7070 2020-05-18T21:40:17.421 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.417+0000 7fa7bb251700 1 -- 172.21.15.80:0/3525042227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6773b0c3-9f49-42a2-b75b-c10c734ff086", "id": 1} v 0) v1 -- 0x7fa7b40569e0 con 0x7fa7b40a7070 2020-05-18T21:40:17.428 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.425+0000 7fa7b9fef700 1 --2- 172.21.15.80:0/3525042227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7b40a7070 0x7fa7b407e790 secure :-1 s=THROTTLE_DONE pgs=14 cs=0 l=1 rx=0x7fa7ac000bc0 tx=0x7fa7ac010920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.429 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.425+0000 7fa7a9ffb700 1 -- 172.21.15.80:0/3525042227 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6773b0c3-9f49-42a2-b75b-c10c734ff086", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa7ac00ae30 con 0x7fa7b40a7070 2020-05-18T21:40:17.429 INFO:teuthology.orchestra.run.smithi080.stdout:1 2020-05-18T21:40:17.430 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.425+0000 7fa7bb251700 1 -- 172.21.15.80:0/3525042227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7b40a7070 msgr2=0x7fa7b407e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.430 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.425+0000 7fa7bb251700 1 --2- 172.21.15.80:0/3525042227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7b40a7070 0x7fa7b407e790 secure :-1 s=READY pgs=14 cs=0 l=1 rx=0x7fa7ac000bc0 tx=0x7fa7ac010920).stop 2020-05-18T21:40:17.431 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.425+0000 7fa7bb251700 1 -- 172.21.15.80:0/3525042227 shutdown_connections 2020-05-18T21:40:17.431 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.425+0000 7fa7bb251700 1 --2- 172.21.15.80:0/3525042227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7b40a7070 0x7fa7b407e790 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.431 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.425+0000 7fa7bb251700 1 --2- 172.21.15.80:0/3525042227 >> v2:172.21.15.196:3300/0 conn(0x7fa7b40a4d70 0x7fa7b409c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.431 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.425+0000 7fa7bb251700 1 -- 172.21.15.80:0/3525042227 >> 172.21.15.80:0/3525042227 conn(0x7fa7b405a7c0 msgr2=0x7fa7b40a5150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:17.431 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.425+0000 7fa7bb251700 1 -- 172.21.15.80:0/3525042227 shutdown_connections 2020-05-18T21:40:17.432 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.425+0000 7fa7bb251700 1 -- 172.21.15.80:0/3525042227 wait complete. 2020-05-18T21:40:17.439 INFO:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new ac1f2e05-9bb7-4768-b712-10e4d2ea4154 2 2020-05-18T21:40:17.539 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60cb658700 1 Processor -- start 2020-05-18T21:40:17.539 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60cb658700 1 -- start start 2020-05-18T21:40:17.539 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60cb658700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60c40a69d0 0x7f60c40a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:17.540 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60cb658700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c40a3eb0 con 0x7f60c40a7070 2020-05-18T21:40:17.540 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60cb658700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c40a4150 con 0x7f60c40a4d70 2020-05-18T21:40:17.540 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60cb658700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c40a5f00 con 0x7f60c40a69d0 2020-05-18T21:40:17.540 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c93f4700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60c40a69d0 0x7f60c40a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.541 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c93f4700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60c40a69d0 0x7f60c40a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33216/0 (socket says 172.21.15.80:33216) 2020-05-18T21:40:17.541 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c9bf5700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f60c40a7070 0x7f60c40a62c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35326/0 (socket says 172.21.15.80:35326) 2020-05-18T21:40:17.541 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c93f4700 1 -- 172.21.15.80:0/1029052875 learned_addr learned my addr 172.21.15.80:0/1029052875 (peer_addr_for_me v2:172.21.15.80:0/0) 2020-05-18T21:40:17.542 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c93f4700 1 -- 172.21.15.80:0/1029052875 >> v1:172.21.15.196:6789/0 conn(0x7f60c40a4d70 legacy=0x7f60c414ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:17.542 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c93f4700 1 -- 172.21.15.80:0/1029052875 >> v1:172.21.15.80:6789/0 conn(0x7f60c40a7070 legacy=0x7f60c40a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.542 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c93f4700 1 -- 172.21.15.80:0/1029052875 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c40a3be0 con 0x7f60c40a69d0 2020-05-18T21:40:17.542 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c8bf3700 1 -- 172.21.15.80:0/1029052875 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083094229 0 0) 0x7f60c40a3eb0 con 0x7f60c40a7070 2020-05-18T21:40:17.542 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c93f4700 1 --2- 172.21.15.80:0/1029052875 >> v2:172.21.15.80:3300/0 conn(0x7f60c40a69d0 0x7f60c40a45f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f60c405b610 tx=0x7f60b8007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.543 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c93f4700 1 --2- 172.21.15.80:0/1029052875 >> v2:172.21.15.80:3300/0 conn(0x7f60c40a69d0 0x7f60c40a45f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f60c405b610 tx=0x7f60b8007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.543 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c93f4700 1 --2- 172.21.15.80:0/1029052875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a69d0 0x7f60c40a45f0 secure :-1 s=READY pgs=16 cs=0 l=1 rx=0x7f60c405b610 tx=0x7f60b8007a80).ready entity=mon.0 client_cookie=46e5e7efa425e90d server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:17.543 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c93f4700 1 --2- 172.21.15.80:0/1029052875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a69d0 0x7f60c40a45f0 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f60c405b610 tx=0x7f60b8007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.543 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c8bf3700 1 -- 172.21.15.80:0/1029052875 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60b8011720 con 0x7f60c40a69d0 2020-05-18T21:40:17.543 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c93f4700 1 --2- 172.21.15.80:0/1029052875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a69d0 0x7f60c40a45f0 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f60c405b610 tx=0x7f60b8007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.543 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c93f4700 1 --2- 172.21.15.80:0/1029052875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a69d0 0x7f60c40a45f0 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f60c405b610 tx=0x7f60b8007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.544 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c8bf3700 1 -- 172.21.15.80:0/1029052875 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b8011880 con 0x7f60c40a69d0 2020-05-18T21:40:17.544 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60c8bf3700 1 -- 172.21.15.80:0/1029052875 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60b8011c20 con 0x7f60c40a69d0 2020-05-18T21:40:17.544 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60cb658700 1 -- 172.21.15.80:0/1029052875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a69d0 msgr2=0x7f60c40a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.544 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60cb658700 1 --2- 172.21.15.80:0/1029052875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a69d0 0x7f60c40a45f0 secure :-1 s=READY pgs=16 cs=0 l=1 rx=0x7f60c405b610 tx=0x7f60b8007a80).stop 2020-05-18T21:40:17.544 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60cb658700 1 -- 172.21.15.80:0/1029052875 shutdown_connections 2020-05-18T21:40:17.545 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60cb658700 1 --2- 172.21.15.80:0/1029052875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a69d0 0x7f60c40a45f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.545 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60cb658700 1 -- 172.21.15.80:0/1029052875 >> 172.21.15.80:0/1029052875 conn(0x7f60c405a7c0 msgr2=0x7f60c405aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:17.545 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60cb658700 1 -- 172.21.15.80:0/1029052875 shutdown_connections 2020-05-18T21:40:17.545 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.533+0000 7f60cb658700 1 -- 172.21.15.80:0/1029052875 wait complete. 2020-05-18T21:40:17.549 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60cb658700 1 Processor -- start 2020-05-18T21:40:17.549 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60cb658700 1 -- start start 2020-05-18T21:40:17.549 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60cb658700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60c40a4d70 0x7f60c4064c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:17.550 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60cb658700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f60c40a69d0 0x7f60c4134d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:17.550 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60cb658700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60c40a7070 0x7f60c41351b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:17.550 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60cb658700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c407f690 con 0x7f60c40a7070 2020-05-18T21:40:17.550 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60cb658700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c40a9300 con 0x7f60c40a69d0 2020-05-18T21:40:17.551 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60cb658700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c4066050 con 0x7f60c40a4d70 2020-05-18T21:40:17.551 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60c9bf5700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60c40a4d70 0x7f60c4064c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.551 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60c40a7070 0x7f60c41351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.553 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60c40a7070 0x7f60c41351b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33222/0 (socket says 172.21.15.80:33222) 2020-05-18T21:40:17.554 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 -- 172.21.15.80:0/1366805316 learned_addr learned my addr 172.21.15.80:0/1366805316 (peer_addr_for_me v2:172.21.15.80:0/0) 2020-05-18T21:40:17.554 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60c93f4700 1 --2- 172.21.15.80:0/1366805316 >> v2:172.21.15.196:3300/0 conn(0x7f60c40a69d0 0x7f60c4134d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.554 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 -- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60c40a4d70 msgr2=0x7f60c4064c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.554 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 --2- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60c40a4d70 0x7f60c4064c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.554 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 -- 172.21.15.80:0/1366805316 >> v2:172.21.15.196:3300/0 conn(0x7f60c40a69d0 msgr2=0x7f60c4134d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.555 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 --2- 172.21.15.80:0/1366805316 >> v2:172.21.15.196:3300/0 conn(0x7f60c40a69d0 0x7f60c4134d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.555 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 -- 172.21.15.80:0/1366805316 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c4134610 con 0x7f60c40a7070 2020-05-18T21:40:17.555 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60c9bf5700 1 --2- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60c40a4d70 0x7f60c4064c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:17.555 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 --2- 172.21.15.80:0/1366805316 >> v2:172.21.15.80:3300/0 conn(0x7f60c40a7070 0x7f60c41351b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f60c4066650 tx=0x7f60bc010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.555 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 --2- 172.21.15.80:0/1366805316 >> v2:172.21.15.80:3300/0 conn(0x7f60c40a7070 0x7f60c41351b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f60c4066650 tx=0x7f60bc010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.556 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 --2- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a7070 0x7f60c41351b0 secure :-1 s=READY pgs=17 cs=0 l=1 rx=0x7f60c4066650 tx=0x7f60bc010c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:17.556 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 --2- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a7070 0x7f60c41351b0 secure :-1 s=THROTTLE_DONE pgs=17 cs=0 l=1 rx=0x7f60c4066650 tx=0x7f60bc010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.556 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60c27fc700 1 -- 172.21.15.80:0/1366805316 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60bc012230 con 0x7f60c40a7070 2020-05-18T21:40:17.556 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 --2- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a7070 0x7f60c41351b0 secure :-1 s=THROTTLE_DONE pgs=17 cs=0 l=1 rx=0x7f60c4066650 tx=0x7f60bc010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.556 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60cb658700 1 -- 172.21.15.80:0/1366805316 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c41348b0 con 0x7f60c40a7070 2020-05-18T21:40:17.556 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 --2- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a7070 0x7f60c41351b0 secure :-1 s=THROTTLE_DONE pgs=17 cs=0 l=1 rx=0x7f60c4066650 tx=0x7f60bc010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.557 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60c27fc700 1 -- 172.21.15.80:0/1366805316 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc012390 con 0x7f60c40a7070 2020-05-18T21:40:17.557 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60c27fc700 1 -- 172.21.15.80:0/1366805316 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60bc012770 con 0x7f60c40a7070 2020-05-18T21:40:17.557 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60cb658700 1 -- 172.21.15.80:0/1366805316 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c4065d00 con 0x7f60c40a7070 2020-05-18T21:40:17.557 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 --2- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a7070 0x7f60c41351b0 secure :-1 s=THROTTLE_DONE pgs=17 cs=0 l=1 rx=0x7f60c4066650 tx=0x7f60bc010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.557 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60c27fc700 1 -- 172.21.15.80:0/1366805316 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) v1 ==== 514+0+0 (secure 0 0 0) 0x7f60bc009270 con 0x7f60c40a7070 2020-05-18T21:40:17.558 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60ca3f6700 1 --2- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a7070 0x7f60c41351b0 secure :-1 s=THROTTLE_DONE pgs=17 cs=0 l=1 rx=0x7f60c4066650 tx=0x7f60bc010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.558 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.537+0000 7f60c27fc700 1 -- 172.21.15.80:0/1366805316 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7f60bc009500 con 0x7f60c40a7070 2020-05-18T21:40:17.558 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.541+0000 7f60cb658700 1 -- 172.21.15.80:0/1366805316 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c40569e0 con 0x7f60c40a7070 2020-05-18T21:40:17.558 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.545+0000 7f60ca3f6700 1 --2- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a7070 0x7f60c41351b0 secure :-1 s=THROTTLE_DONE pgs=17 cs=0 l=1 rx=0x7f60c4066650 tx=0x7f60bc010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.559 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.545+0000 7f60c27fc700 1 -- 172.21.15.80:0/1366805316 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67708 (secure 0 0 0) 0x7f60bc0096c0 con 0x7f60c40a7070 2020-05-18T21:40:17.614 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.609+0000 7f60cb658700 1 -- 172.21.15.80:0/1366805316 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ac1f2e05-9bb7-4768-b712-10e4d2ea4154", "id": 2} v 0) v1 -- 0x7f60c4069060 con 0x7f60c40a7070 2020-05-18T21:40:17.619 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.617+0000 7f60ca3f6700 1 --2- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a7070 0x7f60c41351b0 secure :-1 s=THROTTLE_DONE pgs=17 cs=0 l=1 rx=0x7f60c4066650 tx=0x7f60bc010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.620 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.617+0000 7f60c27fc700 1 -- 172.21.15.80:0/1366805316 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac1f2e05-9bb7-4768-b712-10e4d2ea4154", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f60bc0098a0 con 0x7f60c40a7070 2020-05-18T21:40:17.620 INFO:teuthology.orchestra.run.smithi080.stdout:2 2020-05-18T21:40:17.622 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.617+0000 7f60cb658700 1 -- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a7070 msgr2=0x7f60c41351b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.622 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.617+0000 7f60cb658700 1 --2- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a7070 0x7f60c41351b0 secure :-1 s=READY pgs=17 cs=0 l=1 rx=0x7f60c4066650 tx=0x7f60bc010c50).stop 2020-05-18T21:40:17.622 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.617+0000 7f60cb658700 1 -- 172.21.15.80:0/1366805316 shutdown_connections 2020-05-18T21:40:17.623 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.617+0000 7f60cb658700 1 --2- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60c40a7070 0x7f60c41351b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.623 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.617+0000 7f60cb658700 1 --2- 172.21.15.80:0/1366805316 >> v2:172.21.15.196:3300/0 conn(0x7f60c40a69d0 0x7f60c4134d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.623 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.617+0000 7f60cb658700 1 --2- 172.21.15.80:0/1366805316 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60c40a4d70 0x7f60c4064c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.624 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.617+0000 7f60cb658700 1 -- 172.21.15.80:0/1366805316 >> 172.21.15.80:0/1366805316 conn(0x7f60c405a7c0 msgr2=0x7f60c40a6050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:17.624 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.617+0000 7f60cb658700 1 -- 172.21.15.80:0/1366805316 shutdown_connections 2020-05-18T21:40:17.624 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.617+0000 7f60cb658700 1 -- 172.21.15.80:0/1366805316 wait complete. 2020-05-18T21:40:17.631 INFO:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new bae5ab40-235b-4e0f-8d0d-524afd099b1a 3 2020-05-18T21:40:17.736 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 Processor -- start 2020-05-18T21:40:17.736 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 -- start start 2020-05-18T21:40:17.736 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbe80abb40 0x7ffbe80a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:17.737 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffbe80a38b0 0x7ffbe8024e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:17.737 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe80a6590 con 0x7ffbe80a3470 2020-05-18T21:40:17.737 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe807fec0 con 0x7ffbe80a38b0 2020-05-18T21:40:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80ac3f0 con 0x7ffbe80abb40 2020-05-18T21:40:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbee52f700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbe80abb40 0x7ffbe80a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbee52f700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbe80abb40 0x7ffbe80a7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:37490/0 (socket says 172.21.15.80:37490) 2020-05-18T21:40:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbee52f700 1 -- 172.21.15.80:0/476146923 learned_addr learned my addr 172.21.15.80:0/476146923 (peer_addr_for_me v2:172.21.15.80:0/0) 2020-05-18T21:40:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeff92700 1 --2- 172.21.15.80:0/476146923 >> v2:172.21.15.196:3300/0 conn(0x7ffbe80a38b0 0x7ffbe8024e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.739 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbee52f700 1 -- 172.21.15.80:0/476146923 >> v2:172.21.15.196:3300/0 conn(0x7ffbe80a38b0 msgr2=0x7ffbe8024e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:17.739 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbee52f700 1 --2- 172.21.15.80:0/476146923 >> v2:172.21.15.196:3300/0 conn(0x7ffbe80a38b0 0x7ffbe8024e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:17.739 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbee52f700 1 -- 172.21.15.80:0/476146923 >> v1:172.21.15.196:6789/0 conn(0x7ffbe80a3470 legacy=0x7ffbe8024770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.739 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbee52f700 1 -- 172.21.15.80:0/476146923 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe80a67c0 con 0x7ffbe80abb40 2020-05-18T21:40:17.739 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbee52f700 1 --2- 172.21.15.80:0/476146923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbe80abb40 0x7ffbe80a7ba0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7ffbe4002f70 tx=0x7ffbe4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.740 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbee52f700 1 --2- 172.21.15.80:0/476146923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbe80abb40 0x7ffbe80a7ba0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7ffbe4002f70 tx=0x7ffbe4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.740 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbee52f700 1 --2- 172.21.15.80:0/476146923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbe80abb40 0x7ffbe80a7ba0 secure :-1 s=READY pgs=11 cs=0 l=1 rx=0x7ffbe4002f70 tx=0x7ffbe4007a80).ready entity=mon.2 client_cookie=d948aa197e0fa73d server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:17.740 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbee52f700 1 --2- 172.21.15.80:0/476146923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbe80abb40 0x7ffbe80a7ba0 secure :-1 s=THROTTLE_DONE pgs=11 cs=0 l=1 rx=0x7ffbe4002f70 tx=0x7ffbe4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.740 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbee52f700 1 --2- 172.21.15.80:0/476146923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbe80abb40 0x7ffbe80a7ba0 secure :-1 s=THROTTLE_DONE pgs=11 cs=0 l=1 rx=0x7ffbe4002f70 tx=0x7ffbe4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.740 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbed52d700 1 -- 172.21.15.80:0/476146923 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffbe40116b0 con 0x7ffbe80abb40 2020-05-18T21:40:17.741 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbee52f700 1 --2- 172.21.15.80:0/476146923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbe80abb40 0x7ffbe80a7ba0 secure :-1 s=THROTTLE_DONE pgs=11 cs=0 l=1 rx=0x7ffbe4002f70 tx=0x7ffbe4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.741 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbed52d700 1 -- 172.21.15.80:0/476146923 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe4011810 con 0x7ffbe80abb40 2020-05-18T21:40:17.741 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbed52d700 1 -- 172.21.15.80:0/476146923 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffbe4011c30 con 0x7ffbe80abb40 2020-05-18T21:40:17.742 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 -- 172.21.15.80:0/476146923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbe80abb40 msgr2=0x7ffbe80a7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.742 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 --2- 172.21.15.80:0/476146923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbe80abb40 0x7ffbe80a7ba0 secure :-1 s=READY pgs=11 cs=0 l=1 rx=0x7ffbe4002f70 tx=0x7ffbe4007a80).stop 2020-05-18T21:40:17.742 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 -- 172.21.15.80:0/476146923 shutdown_connections 2020-05-18T21:40:17.742 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 --2- 172.21.15.80:0/476146923 >> v2:172.21.15.196:3300/0 conn(0x7ffbe80a38b0 0x7ffbe8024e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:17.743 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 --2- 172.21.15.80:0/476146923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbe80abb40 0x7ffbe80a7ba0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.743 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 -- 172.21.15.80:0/476146923 >> 172.21.15.80:0/476146923 conn(0x7ffbe8069ac0 msgr2=0x7ffbe8069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:17.743 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 -- 172.21.15.80:0/476146923 shutdown_connections 2020-05-18T21:40:17.743 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.725+0000 7ffbeed30700 1 -- 172.21.15.80:0/476146923 wait complete. 2020-05-18T21:40:17.743 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbeed30700 1 Processor -- start 2020-05-18T21:40:17.744 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbeed30700 1 -- start start 2020-05-18T21:40:17.744 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbeed30700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffbe80a3470 0x7ffbe8129fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:17.744 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbeed30700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffbe80a38b0 0x7ffbe8066290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:17.744 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbeed30700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe812a610 con 0x7ffbe80abb40 2020-05-18T21:40:17.745 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbeed30700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe8065650 con 0x7ffbe80a38b0 2020-05-18T21:40:17.745 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbeed30700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe80671a0 con 0x7ffbe80a3470 2020-05-18T21:40:17.745 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffbe80a38b0 0x7ffbe8066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.745 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffbe80a38b0 0x7ffbe8066290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33232/0 (socket says 172.21.15.80:33232) 2020-05-18T21:40:17.745 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 -- 172.21.15.80:0/1405266130 learned_addr learned my addr 172.21.15.80:0/1405266130 (peer_addr_for_me v2:172.21.15.80:0/0) 2020-05-18T21:40:17.746 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbee52f700 1 --2- 172.21.15.80:0/1405266130 >> v2:172.21.15.196:3300/0 conn(0x7ffbe80a3470 0x7ffbe8129fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.746 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbdeffd700 1 -- 172.21.15.80:0/1405266130 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038827143 0 0) 0x7ffbe812a610 con 0x7ffbe80abb40 2020-05-18T21:40:17.746 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 -- 172.21.15.80:0/1405266130 >> v2:172.21.15.196:3300/0 conn(0x7ffbe80a3470 msgr2=0x7ffbe8129fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.746 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 --2- 172.21.15.80:0/1405266130 >> v2:172.21.15.196:3300/0 conn(0x7ffbe80a3470 0x7ffbe8129fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.746 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 -- 172.21.15.80:0/1405266130 >> v1:172.21.15.196:6789/0 conn(0x7ffbe80abb40 legacy=0x7ffbe8137fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.747 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 -- 172.21.15.80:0/1405266130 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe8066c70 con 0x7ffbe80a38b0 2020-05-18T21:40:17.747 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 --2- 172.21.15.80:0/1405266130 >> v2:172.21.15.80:3300/0 conn(0x7ffbe80a38b0 0x7ffbe8066290 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ffbe80a7310 tx=0x7ffbd800c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.747 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 --2- 172.21.15.80:0/1405266130 >> v2:172.21.15.80:3300/0 conn(0x7ffbe80a38b0 0x7ffbe8066290 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ffbe80a7310 tx=0x7ffbd800c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.747 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 --2- 172.21.15.80:0/1405266130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbe80a38b0 0x7ffbe8066290 secure :-1 s=READY pgs=18 cs=0 l=1 rx=0x7ffbe80a7310 tx=0x7ffbd800c4a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:17.748 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 --2- 172.21.15.80:0/1405266130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbe80a38b0 0x7ffbe8066290 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7ffbe80a7310 tx=0x7ffbd800c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.748 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbdeffd700 1 -- 172.21.15.80:0/1405266130 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffbd800cc60 con 0x7ffbe80a38b0 2020-05-18T21:40:17.748 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 --2- 172.21.15.80:0/1405266130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbe80a38b0 0x7ffbe8066290 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7ffbe80a7310 tx=0x7ffbd800c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.748 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbdeffd700 1 -- 172.21.15.80:0/1405266130 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd800cdc0 con 0x7ffbe80a38b0 2020-05-18T21:40:17.748 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 --2- 172.21.15.80:0/1405266130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbe80a38b0 0x7ffbe8066290 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7ffbe80a7310 tx=0x7ffbd800c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.749 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbdeffd700 1 -- 172.21.15.80:0/1405266130 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffbd8012220 con 0x7ffbe80a38b0 2020-05-18T21:40:17.749 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbeed30700 1 -- 172.21.15.80:0/1405266130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe8064bd0 con 0x7ffbe80a38b0 2020-05-18T21:40:17.749 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbeed30700 1 -- 172.21.15.80:0/1405266130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe8137960 con 0x7ffbe80a38b0 2020-05-18T21:40:17.749 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 --2- 172.21.15.80:0/1405266130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbe80a38b0 0x7ffbe8066290 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7ffbe80a7310 tx=0x7ffbd800c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.749 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbdeffd700 1 -- 172.21.15.80:0/1405266130 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) v1 ==== 514+0+0 (secure 0 0 0) 0x7ffbd8006a20 con 0x7ffbe80a38b0 2020-05-18T21:40:17.750 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbedd2e700 1 --2- 172.21.15.80:0/1405266130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbe80a38b0 0x7ffbe8066290 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7ffbe80a7310 tx=0x7ffbd800c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.750 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.729+0000 7ffbdeffd700 1 -- 172.21.15.80:0/1405266130 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7ffbd8012410 con 0x7ffbe80a38b0 2020-05-18T21:40:17.751 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.737+0000 7ffbeed30700 1 -- 172.21.15.80:0/1405266130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe80569e0 con 0x7ffbe80a38b0 2020-05-18T21:40:17.751 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.741+0000 7ffbedd2e700 1 --2- 172.21.15.80:0/1405266130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbe80a38b0 0x7ffbe8066290 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7ffbe80a7310 tx=0x7ffbd800c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.752 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.741+0000 7ffbdeffd700 1 -- 172.21.15.80:0/1405266130 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67708 (secure 0 0 0) 0x7ffbd8006cb0 con 0x7ffbe80a38b0 2020-05-18T21:40:17.810 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.805+0000 7ffbeed30700 1 -- 172.21.15.80:0/1405266130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bae5ab40-235b-4e0f-8d0d-524afd099b1a", "id": 3} v 0) v1 -- 0x7ffbe805c940 con 0x7ffbe80a38b0 2020-05-18T21:40:17.816 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.813+0000 7ffbedd2e700 1 --2- 172.21.15.80:0/1405266130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbe80a38b0 0x7ffbe8066290 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7ffbe80a7310 tx=0x7ffbd800c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.817 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.813+0000 7ffbdeffd700 1 -- 172.21.15.80:0/1405266130 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bae5ab40-235b-4e0f-8d0d-524afd099b1a", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffbd8010220 con 0x7ffbe80a38b0 2020-05-18T21:40:17.817 INFO:teuthology.orchestra.run.smithi080.stdout:3 2020-05-18T21:40:17.818 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.813+0000 7ffbeed30700 1 -- 172.21.15.80:0/1405266130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbe80a38b0 msgr2=0x7ffbe8066290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.818 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.813+0000 7ffbeed30700 1 --2- 172.21.15.80:0/1405266130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbe80a38b0 0x7ffbe8066290 secure :-1 s=READY pgs=18 cs=0 l=1 rx=0x7ffbe80a7310 tx=0x7ffbd800c4a0).stop 2020-05-18T21:40:17.819 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.813+0000 7ffbeed30700 1 -- 172.21.15.80:0/1405266130 shutdown_connections 2020-05-18T21:40:17.820 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.813+0000 7ffbeed30700 1 --2- 172.21.15.80:0/1405266130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbe80a38b0 0x7ffbe8066290 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.820 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.813+0000 7ffbeed30700 1 --2- 172.21.15.80:0/1405266130 >> v2:172.21.15.196:3300/0 conn(0x7ffbe80a3470 0x7ffbe8129fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.820 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.813+0000 7ffbeed30700 1 -- 172.21.15.80:0/1405266130 >> 172.21.15.80:0/1405266130 conn(0x7ffbe8069ac0 msgr2=0x7ffbe80a8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:17.820 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.813+0000 7ffbeed30700 1 -- 172.21.15.80:0/1405266130 shutdown_connections 2020-05-18T21:40:17.821 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.813+0000 7ffbeed30700 1 -- 172.21.15.80:0/1405266130 wait complete. 2020-05-18T21:40:17.829 INFO:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 275fc52c-58e4-4f35-a0cb-e117fe36bf5c 4 2020-05-18T21:40:17.928 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0417fff700 1 Processor -- start 2020-05-18T21:40:17.928 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0417fff700 1 -- start start 2020-05-18T21:40:17.929 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0417fff700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04180a7ba0 0x7f041814c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:17.929 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0417fff700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04180a6590 con 0x7f04180abb40 2020-05-18T21:40:17.929 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0417fff700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04180ac3f0 con 0x7f041814cb00 2020-05-18T21:40:17.929 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0417fff700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041807fec0 con 0x7f04180a7ba0 2020-05-18T21:40:17.930 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0416ffd700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04180a7ba0 0x7f041814c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.930 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f04177fe700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f04180abb40 0x7f04180a3470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35350/0 (socket says 172.21.15.80:35350) 2020-05-18T21:40:17.930 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f04177fe700 1 -- 172.21.15.80:0/4248100282 learned_addr learned my addr 172.21.15.80:0/4248100282 (peer_addr_for_me v1:172.21.15.80:0/0) 2020-05-18T21:40:17.930 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f04167fc700 1 -- 172.21.15.80:0/4248100282 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851630384 0 0) 0x7f04180a6590 con 0x7f04180abb40 2020-05-18T21:40:17.931 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f04167fc700 1 -- 172.21.15.80:0/4248100282 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0400003270 con 0x7f04180abb40 2020-05-18T21:40:17.931 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0416ffd700 1 -- 172.21.15.80:0/4248100282 >> v1:172.21.15.196:6789/0 conn(0x7f041814cb00 legacy=0x7f041814eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.931 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0416ffd700 1 -- 172.21.15.80:0/4248100282 >> v1:172.21.15.80:6789/0 conn(0x7f04180abb40 legacy=0x7f04180a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.931 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0416ffd700 1 -- 172.21.15.80:0/4248100282 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04180a67c0 con 0x7f04180a7ba0 2020-05-18T21:40:17.932 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0416ffd700 1 --2- 172.21.15.80:0/4248100282 >> v2:172.21.15.80:3300/0 conn(0x7f04180a7ba0 0x7f041814c6a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f040c002f70 tx=0x7f040c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.932 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0416ffd700 1 --2- 172.21.15.80:0/4248100282 >> v2:172.21.15.80:3300/0 conn(0x7f04180a7ba0 0x7f041814c6a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f040c002f70 tx=0x7f040c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.932 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0416ffd700 1 --2- 172.21.15.80:0/4248100282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04180a7ba0 0x7f041814c6a0 secure :-1 s=READY pgs=20 cs=0 l=1 rx=0x7f040c002f70 tx=0x7f040c007a80).ready entity=mon.0 client_cookie=eac2ab6b7a015284 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:17.932 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0416ffd700 1 --2- 172.21.15.80:0/4248100282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04180a7ba0 0x7f041814c6a0 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7f040c002f70 tx=0x7f040c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.933 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0416ffd700 1 --2- 172.21.15.80:0/4248100282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04180a7ba0 0x7f041814c6a0 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7f040c002f70 tx=0x7f040c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.933 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f04167fc700 1 -- 172.21.15.80:0/4248100282 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f040c008e80 con 0x7f04180a7ba0 2020-05-18T21:40:17.933 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0416ffd700 1 --2- 172.21.15.80:0/4248100282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04180a7ba0 0x7f041814c6a0 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7f040c002f70 tx=0x7f040c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.933 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f04167fc700 1 -- 172.21.15.80:0/4248100282 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c0115f0 con 0x7f04180a7ba0 2020-05-18T21:40:17.934 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0417fff700 1 -- 172.21.15.80:0/4248100282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04180a7ba0 msgr2=0x7f041814c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.934 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f0417fff700 1 --2- 172.21.15.80:0/4248100282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04180a7ba0 0x7f041814c6a0 secure :-1 s=READY pgs=20 cs=0 l=1 rx=0x7f040c002f70 tx=0x7f040c007a80).stop 2020-05-18T21:40:17.934 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.921+0000 7f04167fc700 1 -- 172.21.15.80:0/4248100282 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f040c0119d0 con 0x7f04180a7ba0 2020-05-18T21:40:17.935 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0417fff700 1 -- 172.21.15.80:0/4248100282 shutdown_connections 2020-05-18T21:40:17.935 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0417fff700 1 --2- 172.21.15.80:0/4248100282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04180a7ba0 0x7f041814c6a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.935 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0417fff700 1 -- 172.21.15.80:0/4248100282 >> 172.21.15.80:0/4248100282 conn(0x7f0418069ac0 msgr2=0x7f0418069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:17.935 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0417fff700 1 -- 172.21.15.80:0/4248100282 shutdown_connections 2020-05-18T21:40:17.936 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0417fff700 1 -- 172.21.15.80:0/4248100282 wait complete. 2020-05-18T21:40:17.939 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0417fff700 1 Processor -- start 2020-05-18T21:40:17.939 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0417fff700 1 -- start start 2020-05-18T21:40:17.939 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0417fff700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04180abb40 0x7f041814f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:17.940 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0417fff700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:17.940 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0417fff700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0418062480 con 0x7f04180a7ba0 2020-05-18T21:40:17.940 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0417fff700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04180626e0 con 0x7f04180abb40 2020-05-18T21:40:17.940 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0417fff700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0418062980 con 0x7f041814cb00 2020-05-18T21:40:17.941 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0416ffd700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04180abb40 0x7f041814f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.941 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0416ffd700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04180abb40 0x7f041814f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33246/0 (socket says 172.21.15.80:33246) 2020-05-18T21:40:17.941 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f0416ffd700 1 -- 172.21.15.80:0/2641743432 learned_addr learned my addr 172.21.15.80:0/2641743432 (peer_addr_for_me v2:172.21.15.80:0/0) 2020-05-18T21:40:17.941 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f041d7ef700 1 --2- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:17.942 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f041d7ef700 1 -- 172.21.15.80:0/2641743432 >> v2:172.21.15.80:3300/0 conn(0x7f04180abb40 msgr2=0x7f041814f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.942 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f041d7ef700 1 --2- 172.21.15.80:0/2641743432 >> v2:172.21.15.80:3300/0 conn(0x7f04180abb40 0x7f041814f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:17.942 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f041d7ef700 1 -- 172.21.15.80:0/2641743432 >> v1:172.21.15.196:6789/0 conn(0x7f04180a7ba0 legacy=0x7f04180640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:17.943 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f041d7ef700 1 -- 172.21.15.80:0/2641743432 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04180647d0 con 0x7f041814cb00 2020-05-18T21:40:17.943 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f041d7ef700 1 --2- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0418063fa0 tx=0x7f0410010be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.943 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f041d7ef700 1 --2- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0418063fa0 tx=0x7f0410010be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.943 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.925+0000 7f041d7ef700 1 --2- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 secure :-1 s=READY pgs=12 cs=0 l=1 rx=0x7f0418063fa0 tx=0x7f0410010be0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:17.944 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.929+0000 7f041d7ef700 1 --2- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f0418063fa0 tx=0x7f0410010be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.944 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.929+0000 7f03fffff700 1 -- 172.21.15.80:0/2641743432 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0410002280 con 0x7f041814cb00 2020-05-18T21:40:17.944 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.929+0000 7f041d7ef700 1 --2- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f0418063fa0 tx=0x7f0410010be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.944 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.929+0000 7f041d7ef700 1 --2- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f0418063fa0 tx=0x7f0410010be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.945 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.929+0000 7f03fffff700 1 -- 172.21.15.80:0/2641743432 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04100023e0 con 0x7f041814cb00 2020-05-18T21:40:17.945 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.929+0000 7f0417fff700 1 -- 172.21.15.80:0/2641743432 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04180a3c40 con 0x7f041814cb00 2020-05-18T21:40:17.945 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.929+0000 7f03fffff700 1 -- 172.21.15.80:0/2641743432 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0410002800 con 0x7f041814cb00 2020-05-18T21:40:17.945 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.929+0000 7f0417fff700 1 -- 172.21.15.80:0/2641743432 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0418122360 con 0x7f041814cb00 2020-05-18T21:40:17.946 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.929+0000 7f041d7ef700 1 --2- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f0418063fa0 tx=0x7f0410010be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.946 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.929+0000 7f03fffff700 1 -- 172.21.15.80:0/2641743432 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 1) v1 ==== 514+0+0 (secure 0 0 0) 0x7f041000b2c0 con 0x7f041814cb00 2020-05-18T21:40:17.946 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.929+0000 7f041d7ef700 1 --2- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f0418063fa0 tx=0x7f0410010be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.946 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.929+0000 7f03fffff700 1 -- 172.21.15.80:0/2641743432 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f041000b550 con 0x7f041814cb00 2020-05-18T21:40:17.946 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.929+0000 7f0417fff700 1 -- 172.21.15.80:0/2641743432 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04180569e0 con 0x7f041814cb00 2020-05-18T21:40:17.947 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.933+0000 7f041d7ef700 1 --2- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f0418063fa0 tx=0x7f0410010be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:17.947 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:17.933+0000 7f03fffff700 1 -- 172.21.15.80:0/2641743432 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67708 (secure 0 0 0) 0x7f041000b7f0 con 0x7f041814cb00 2020-05-18T21:40:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.001+0000 7f0417fff700 1 -- 172.21.15.80:0/2641743432 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd new", "uuid": "275fc52c-58e4-4f35-a0cb-e117fe36bf5c", "id": 4} v 0) v1 -- 0x7f041805c940 con 0x7f041814cb00 2020-05-18T21:40:18.010 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.005+0000 7f041d7ef700 1 --2- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f0418063fa0 tx=0x7f0410010be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.010 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.005+0000 7f03fffff700 1 -- 172.21.15.80:0/2641743432 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "275fc52c-58e4-4f35-a0cb-e117fe36bf5c", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f041000b990 con 0x7f041814cb00 2020-05-18T21:40:18.011 INFO:teuthology.orchestra.run.smithi080.stdout:4 2020-05-18T21:40:18.012 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.009+0000 7f0417fff700 1 -- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 msgr2=0x7f041807e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.012 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.009+0000 7f0417fff700 1 --2- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 secure :-1 s=READY pgs=12 cs=0 l=1 rx=0x7f0418063fa0 tx=0x7f0410010be0).stop 2020-05-18T21:40:18.012 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.009+0000 7f0417fff700 1 -- 172.21.15.80:0/2641743432 shutdown_connections 2020-05-18T21:40:18.013 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.009+0000 7f0417fff700 1 --2- 172.21.15.80:0/2641743432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f041814cb00 0x7f041807e450 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:18.013 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.009+0000 7f0417fff700 1 --2- 172.21.15.80:0/2641743432 >> v2:172.21.15.80:3300/0 conn(0x7f04180abb40 0x7f041814f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:18.013 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.009+0000 7f0417fff700 1 -- 172.21.15.80:0/2641743432 >> 172.21.15.80:0/2641743432 conn(0x7f0418069ac0 msgr2=0x7f041805eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:18.013 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.009+0000 7f0417fff700 1 -- 172.21.15.80:0/2641743432 shutdown_connections 2020-05-18T21:40:18.014 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.009+0000 7f0417fff700 1 -- 172.21.15.80:0/2641743432 wait complete. 2020-05-18T21:40:18.021 INFO:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 9795b89c-e2f8-4abf-b9c7-796727a57002 5 2020-05-18T21:40:18.123 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 Processor -- start 2020-05-18T21:40:18.123 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 -- start start 2020-05-18T21:40:18.124 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd2900abb40 0x7fd2900a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:18.124 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2900a3470 0x7fd290024770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:18.124 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd2900a38b0 0x7fd290024bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:18.124 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29007fec0 con 0x7fd2900abb40 2020-05-18T21:40:18.124 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2900a6590 con 0x7fd2900a38b0 2020-05-18T21:40:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2900ac3f0 con 0x7fd2900a3470 2020-05-18T21:40:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd2956fd700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd2900abb40 0x7fd2900a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd294efc700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2900a3470 0x7fd290024770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd294efc700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2900a3470 0x7fd290024770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:37516/0 (socket says 172.21.15.80:37516) 2020-05-18T21:40:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd294efc700 1 -- 172.21.15.80:0/4028622330 learned_addr learned my addr 172.21.15.80:0/4028622330 (peer_addr_for_me v2:172.21.15.80:0/0) 2020-05-18T21:40:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd2956fd700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd2900abb40 0x7fd2900a7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33250/0 (socket says 172.21.15.80:33250) 2020-05-18T21:40:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd297160700 1 --2- 172.21.15.80:0/4028622330 >> v2:172.21.15.196:3300/0 conn(0x7fd2900a38b0 0x7fd290024bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd2956fd700 1 -- 172.21.15.80:0/4028622330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2900a3470 msgr2=0x7fd290024770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd2956fd700 1 --2- 172.21.15.80:0/4028622330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2900a3470 0x7fd290024770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd2956fd700 1 -- 172.21.15.80:0/4028622330 >> v2:172.21.15.196:3300/0 conn(0x7fd2900a38b0 msgr2=0x7fd290024bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd2956fd700 1 --2- 172.21.15.80:0/4028622330 >> v2:172.21.15.196:3300/0 conn(0x7fd2900a38b0 0x7fd290024bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:18.128 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd2956fd700 1 -- 172.21.15.80:0/4028622330 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900a67c0 con 0x7fd2900abb40 2020-05-18T21:40:18.128 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd294efc700 1 --2- 172.21.15.80:0/4028622330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2900a3470 0x7fd290024770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:18.128 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd297160700 1 --2- 172.21.15.80:0/4028622330 >> v2:172.21.15.196:3300/0 conn(0x7fd2900a38b0 0x7fd290024bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_reply_more state changed! 2020-05-18T21:40:18.128 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd2956fd700 1 --2- 172.21.15.80:0/4028622330 >> v2:172.21.15.80:3300/0 conn(0x7fd2900abb40 0x7fd2900a7ba0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fd290080000 tx=0x7fd288008ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.128 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd2956fd700 1 --2- 172.21.15.80:0/4028622330 >> v2:172.21.15.80:3300/0 conn(0x7fd2900abb40 0x7fd2900a7ba0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fd290080000 tx=0x7fd288008ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd2956fd700 1 --2- 172.21.15.80:0/4028622330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd2900a7ba0 secure :-1 s=READY pgs=21 cs=0 l=1 rx=0x7fd290080000 tx=0x7fd288008ca0).ready entity=mon.0 client_cookie=49e11e6a8da49caa server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd2956fd700 1 --2- 172.21.15.80:0/4028622330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd2900a7ba0 secure :-1 s=THROTTLE_DONE pgs=21 cs=0 l=1 rx=0x7fd290080000 tx=0x7fd288008ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd287fff700 1 -- 172.21.15.80:0/4028622330 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd288011e60 con 0x7fd2900abb40 2020-05-18T21:40:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd2956fd700 1 --2- 172.21.15.80:0/4028622330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd2900a7ba0 secure :-1 s=THROTTLE_DONE pgs=21 cs=0 l=1 rx=0x7fd290080000 tx=0x7fd288008ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd2956fd700 1 --2- 172.21.15.80:0/4028622330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd2900a7ba0 secure :-1 s=THROTTLE_DONE pgs=21 cs=0 l=1 rx=0x7fd290080000 tx=0x7fd288008ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.130 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd287fff700 1 -- 172.21.15.80:0/4028622330 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28800a180 con 0x7fd2900abb40 2020-05-18T21:40:18.130 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd287fff700 1 -- 172.21.15.80:0/4028622330 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd28800a560 con 0x7fd2900abb40 2020-05-18T21:40:18.130 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 -- 172.21.15.80:0/4028622330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 msgr2=0x7fd2900a7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.130 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 --2- 172.21.15.80:0/4028622330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd2900a7ba0 secure :-1 s=READY pgs=21 cs=0 l=1 rx=0x7fd290080000 tx=0x7fd288008ca0).stop 2020-05-18T21:40:18.130 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 -- 172.21.15.80:0/4028622330 shutdown_connections 2020-05-18T21:40:18.130 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 --2- 172.21.15.80:0/4028622330 >> v2:172.21.15.196:3300/0 conn(0x7fd2900a38b0 0x7fd290024bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:18.131 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 --2- 172.21.15.80:0/4028622330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2900a3470 0x7fd290024770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:18.131 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 --2- 172.21.15.80:0/4028622330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd2900a7ba0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:18.131 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 -- 172.21.15.80:0/4028622330 >> 172.21.15.80:0/4028622330 conn(0x7fd290069ac0 msgr2=0x7fd290069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:18.131 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 -- 172.21.15.80:0/4028622330 shutdown_connections 2020-05-18T21:40:18.131 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.113+0000 7fd295efe700 1 -- 172.21.15.80:0/4028622330 wait complete. 2020-05-18T21:40:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd295efe700 1 Processor -- start 2020-05-18T21:40:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd295efe700 1 -- start start 2020-05-18T21:40:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd295efe700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd2900abb40 0x7fd290066090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd295efe700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29013bbf0 con 0x7fd2900a3470 2020-05-18T21:40:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd295efe700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29013be20 con 0x7fd2900a38b0 2020-05-18T21:40:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd295efe700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29013c0c0 con 0x7fd2900abb40 2020-05-18T21:40:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd297160700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd2900abb40 0x7fd290066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd2956fd700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd2900a3470 0x7fd29013c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35368/0 (socket says 172.21.15.80:35368) 2020-05-18T21:40:18.134 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd2956fd700 1 -- 172.21.15.80:0/1228511657 learned_addr learned my addr 172.21.15.80:0/1228511657 (peer_addr_for_me v1:172.21.15.80:0/0) 2020-05-18T21:40:18.134 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd297160700 1 -- 172.21.15.80:0/1228511657 >> v1:172.21.15.196:6789/0 conn(0x7fd2900a38b0 legacy=0x7fd29013af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.134 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd297160700 1 -- 172.21.15.80:0/1228511657 >> v1:172.21.15.80:6789/0 conn(0x7fd2900a3470 legacy=0x7fd29013c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.134 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd285ffb700 1 -- 172.21.15.80:0/1228511657 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677290384 0 0) 0x7fd29013bbf0 con 0x7fd2900a3470 2020-05-18T21:40:18.134 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd297160700 1 -- 172.21.15.80:0/1228511657 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900665a0 con 0x7fd2900abb40 2020-05-18T21:40:18.135 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd297160700 1 --2- 172.21.15.80:0/1228511657 >> v2:172.21.15.80:3300/0 conn(0x7fd2900abb40 0x7fd290066090 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fd28c002f70 tx=0x7fd28c010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.135 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd297160700 1 --2- 172.21.15.80:0/1228511657 >> v2:172.21.15.80:3300/0 conn(0x7fd2900abb40 0x7fd290066090 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fd28c002f70 tx=0x7fd28c010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.135 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd297160700 1 --2- 172.21.15.80:0/1228511657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd290066090 secure :-1 s=READY pgs=23 cs=0 l=1 rx=0x7fd28c002f70 tx=0x7fd28c010a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:18.135 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd297160700 1 --2- 172.21.15.80:0/1228511657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd290066090 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7fd28c002f70 tx=0x7fd28c010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.135 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd285ffb700 1 -- 172.21.15.80:0/1228511657 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd28c013070 con 0x7fd2900abb40 2020-05-18T21:40:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd297160700 1 --2- 172.21.15.80:0/1228511657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd290066090 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7fd28c002f70 tx=0x7fd28c010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd297160700 1 --2- 172.21.15.80:0/1228511657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd290066090 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7fd28c002f70 tx=0x7fd28c010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd285ffb700 1 -- 172.21.15.80:0/1228511657 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c007180 con 0x7fd2900abb40 2020-05-18T21:40:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd295efe700 1 -- 172.21.15.80:0/1228511657 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2900a2ab0 con 0x7fd2900abb40 2020-05-18T21:40:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd285ffb700 1 -- 172.21.15.80:0/1228511657 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd28c007520 con 0x7fd2900abb40 2020-05-18T21:40:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd295efe700 1 -- 172.21.15.80:0/1228511657 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd290070290 con 0x7fd2900abb40 2020-05-18T21:40:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd297160700 1 --2- 172.21.15.80:0/1228511657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd290066090 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7fd28c002f70 tx=0x7fd28c010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd297160700 1 --2- 172.21.15.80:0/1228511657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd290066090 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7fd28c002f70 tx=0x7fd28c010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd285ffb700 1 -- 172.21.15.80:0/1228511657 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) v1 ==== 514+0+0 (secure 0 0 0) 0x7fd28c012630 con 0x7fd2900abb40 2020-05-18T21:40:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.117+0000 7fd285ffb700 1 -- 172.21.15.80:0/1228511657 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7fd28c006d20 con 0x7fd2900abb40 2020-05-18T21:40:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.121+0000 7fd295efe700 1 -- 172.21.15.80:0/1228511657 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2900569e0 con 0x7fd2900abb40 2020-05-18T21:40:18.138 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.121+0000 7fd297160700 1 --2- 172.21.15.80:0/1228511657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd290066090 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7fd28c002f70 tx=0x7fd28c010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.138 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.121+0000 7fd285ffb700 1 -- 172.21.15.80:0/1228511657 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67708 (secure 0 0 0) 0x7fd28c012af0 con 0x7fd2900abb40 2020-05-18T21:40:18.193 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.189+0000 7fd295efe700 1 -- 172.21.15.80:0/1228511657 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9795b89c-e2f8-4abf-b9c7-796727a57002", "id": 5} v 0) v1 -- 0x7fd29013c230 con 0x7fd2900abb40 2020-05-18T21:40:18.198 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.193+0000 7fd297160700 1 --2- 172.21.15.80:0/1228511657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd290066090 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7fd28c002f70 tx=0x7fd28c010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.198 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.193+0000 7fd285ffb700 1 -- 172.21.15.80:0/1228511657 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9795b89c-e2f8-4abf-b9c7-796727a57002", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd28c012d70 con 0x7fd2900abb40 2020-05-18T21:40:18.199 INFO:teuthology.orchestra.run.smithi080.stdout:5 2020-05-18T21:40:18.199 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.197+0000 7fd295efe700 1 -- 172.21.15.80:0/1228511657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 msgr2=0x7fd290066090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.200 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.197+0000 7fd295efe700 1 --2- 172.21.15.80:0/1228511657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd290066090 secure :-1 s=READY pgs=23 cs=0 l=1 rx=0x7fd28c002f70 tx=0x7fd28c010a90).stop 2020-05-18T21:40:18.200 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.197+0000 7fd295efe700 1 -- 172.21.15.80:0/1228511657 shutdown_connections 2020-05-18T21:40:18.200 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.197+0000 7fd295efe700 1 --2- 172.21.15.80:0/1228511657 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2900abb40 0x7fd290066090 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:18.200 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.197+0000 7fd295efe700 1 -- 172.21.15.80:0/1228511657 >> 172.21.15.80:0/1228511657 conn(0x7fd290069ac0 msgr2=0x7fd29005eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:18.200 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.197+0000 7fd295efe700 1 -- 172.21.15.80:0/1228511657 shutdown_connections 2020-05-18T21:40:18.201 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.197+0000 7fd295efe700 1 -- 172.21.15.80:0/1228511657 wait complete. 2020-05-18T21:40:18.210 INFO:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new e42df2e8-9cc5-4c7e-a243-bd18ccfbd002 6 2020-05-18T21:40:18.307 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3cb59e700 1 Processor -- start 2020-05-18T21:40:18.308 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3cb59e700 1 -- start start 2020-05-18T21:40:18.308 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3cb59e700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa3cc0a7ba0 0x7fa3cc14c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:18.308 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3cb59e700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa3cc14cb00 0x7fa3cc14eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:18.308 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3cb59e700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc0a6590 con 0x7fa3cc0abb40 2020-05-18T21:40:18.308 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3cb59e700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3cc07fec0 con 0x7fa3cc14cb00 2020-05-18T21:40:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3cb59e700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3cc0ac3f0 con 0x7fa3cc0a7ba0 2020-05-18T21:40:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3d0a7a700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa3cc14cb00 0x7fa3cc14eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3d0a7a700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa3cc14cb00 0x7fa3cc14eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33264/0 (socket says 172.21.15.80:33264) 2020-05-18T21:40:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3d0a7a700 1 -- 172.21.15.80:0/2141269423 learned_addr learned my addr 172.21.15.80:0/2141269423 (peer_addr_for_me v2:172.21.15.80:0/0) 2020-05-18T21:40:18.310 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3cad9d700 1 --1- 172.21.15.80:0/2141269423 >> v1:172.21.15.80:6789/0 conn(0x7fa3cc0abb40 0x7fa3cc0a3470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35374/0 (socket says 172.21.15.80:35374) 2020-05-18T21:40:18.310 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3ca59c700 1 --2- 172.21.15.80:0/2141269423 >> v2:172.21.15.196:3300/0 conn(0x7fa3cc0a7ba0 0x7fa3cc14c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:18.310 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3d0a7a700 1 -- 172.21.15.80:0/2141269423 >> v2:172.21.15.196:3300/0 conn(0x7fa3cc0a7ba0 msgr2=0x7fa3cc14c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:18.311 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3d0a7a700 1 --2- 172.21.15.80:0/2141269423 >> v2:172.21.15.196:3300/0 conn(0x7fa3cc0a7ba0 0x7fa3cc14c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:18.311 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3d0a7a700 1 -- 172.21.15.80:0/2141269423 >> v1:172.21.15.80:6789/0 conn(0x7fa3cc0abb40 legacy=0x7fa3cc0a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.311 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3d0a7a700 1 -- 172.21.15.80:0/2141269423 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc0a67c0 con 0x7fa3cc14cb00 2020-05-18T21:40:18.312 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3c9d9b700 1 -- 172.21.15.80:0/2141269423 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 726707002 0 0) 0x7fa3cc0a6590 con 0x7fa3cc0abb40 2020-05-18T21:40:18.312 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3d0a7a700 1 --2- 172.21.15.80:0/2141269423 >> v2:172.21.15.80:3300/0 conn(0x7fa3cc14cb00 0x7fa3cc14eef0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fa3bc002f70 tx=0x7fa3bc007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.312 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3d0a7a700 1 --2- 172.21.15.80:0/2141269423 >> v2:172.21.15.80:3300/0 conn(0x7fa3cc14cb00 0x7fa3cc14eef0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fa3bc002f70 tx=0x7fa3bc007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.313 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3d0a7a700 1 --2- 172.21.15.80:0/2141269423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc14cb00 0x7fa3cc14eef0 secure :-1 s=READY pgs=25 cs=0 l=1 rx=0x7fa3bc002f70 tx=0x7fa3bc007a80).ready entity=mon.0 client_cookie=681af7d0649d06cd server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:18.313 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3d0a7a700 1 --2- 172.21.15.80:0/2141269423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc14cb00 0x7fa3cc14eef0 secure :-1 s=THROTTLE_DONE pgs=25 cs=0 l=1 rx=0x7fa3bc002f70 tx=0x7fa3bc007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.313 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3c9d9b700 1 -- 172.21.15.80:0/2141269423 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3bc011760 con 0x7fa3cc14cb00 2020-05-18T21:40:18.314 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3d0a7a700 1 --2- 172.21.15.80:0/2141269423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc14cb00 0x7fa3cc14eef0 secure :-1 s=THROTTLE_DONE pgs=25 cs=0 l=1 rx=0x7fa3bc002f70 tx=0x7fa3bc007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.314 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3d0a7a700 1 --2- 172.21.15.80:0/2141269423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc14cb00 0x7fa3cc14eef0 secure :-1 s=THROTTLE_DONE pgs=25 cs=0 l=1 rx=0x7fa3bc002f70 tx=0x7fa3bc007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.314 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3c9d9b700 1 -- 172.21.15.80:0/2141269423 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc0118c0 con 0x7fa3cc14cb00 2020-05-18T21:40:18.315 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3c9d9b700 1 -- 172.21.15.80:0/2141269423 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3bc011ca0 con 0x7fa3cc14cb00 2020-05-18T21:40:18.315 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3cb59e700 1 -- 172.21.15.80:0/2141269423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc14cb00 msgr2=0x7fa3cc14eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.315 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.301+0000 7fa3cb59e700 1 --2- 172.21.15.80:0/2141269423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc14cb00 0x7fa3cc14eef0 secure :-1 s=READY pgs=25 cs=0 l=1 rx=0x7fa3bc002f70 tx=0x7fa3bc007a80).stop 2020-05-18T21:40:18.316 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 -- 172.21.15.80:0/2141269423 shutdown_connections 2020-05-18T21:40:18.316 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 --2- 172.21.15.80:0/2141269423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc14cb00 0x7fa3cc14eef0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:18.316 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 --2- 172.21.15.80:0/2141269423 >> v2:172.21.15.196:3300/0 conn(0x7fa3cc0a7ba0 0x7fa3cc14c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:18.316 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 -- 172.21.15.80:0/2141269423 >> 172.21.15.80:0/2141269423 conn(0x7fa3cc069ac0 msgr2=0x7fa3cc069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:18.317 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 -- 172.21.15.80:0/2141269423 shutdown_connections 2020-05-18T21:40:18.317 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 -- 172.21.15.80:0/2141269423 wait complete. 2020-05-18T21:40:18.321 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 Processor -- start 2020-05-18T21:40:18.322 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 -- start start 2020-05-18T21:40:18.322 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:18.322 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc065690 con 0x7fa3cc14cb00 2020-05-18T21:40:18.323 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc065930 con 0x7fa3cc0a7ba0 2020-05-18T21:40:18.323 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3cc07f250 con 0x7fa3cc0abb40 2020-05-18T21:40:18.323 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3ca59c700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:18.324 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3d0a7a700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa3cc14cb00 0x7fa3cc12f8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35384/0 (socket says 172.21.15.80:35384) 2020-05-18T21:40:18.324 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3ca59c700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33270/0 (socket says 172.21.15.80:33270) 2020-05-18T21:40:18.324 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3d0a7a700 1 -- 172.21.15.80:0/118865453 learned_addr learned my addr 172.21.15.80:0/118865453 (peer_addr_for_me v1:172.21.15.80:0/0) 2020-05-18T21:40:18.325 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3bb7fe700 1 -- 172.21.15.80:0/118865453 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337803245 0 0) 0x7fa3cc065690 con 0x7fa3cc14cb00 2020-05-18T21:40:18.325 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3ca59c700 1 -- 172.21.15.80:0/118865453 >> v1:172.21.15.196:6789/0 conn(0x7fa3cc0a7ba0 legacy=0x7fa3cc07eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.325 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3ca59c700 1 -- 172.21.15.80:0/118865453 >> v1:172.21.15.80:6789/0 conn(0x7fa3cc14cb00 legacy=0x7fa3cc12f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.325 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3ca59c700 1 -- 172.21.15.80:0/118865453 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc0653c0 con 0x7fa3cc0abb40 2020-05-18T21:40:18.326 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3ca59c700 1 --2- 172.21.15.80:0/118865453 >> v2:172.21.15.80:3300/0 conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa3c0002f70 tx=0x7fa3c0010ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.326 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3ca59c700 1 --2- 172.21.15.80:0/118865453 >> v2:172.21.15.80:3300/0 conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa3c0002f70 tx=0x7fa3c0010ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.326 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3ca59c700 1 --2- 172.21.15.80:0/118865453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 secure :-1 s=READY pgs=27 cs=0 l=1 rx=0x7fa3c0002f70 tx=0x7fa3c0010ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:18.326 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3ca59c700 1 --2- 172.21.15.80:0/118865453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 secure :-1 s=THROTTLE_DONE pgs=27 cs=0 l=1 rx=0x7fa3c0002f70 tx=0x7fa3c0010ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.326 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3bb7fe700 1 -- 172.21.15.80:0/118865453 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3c0007180 con 0x7fa3cc0abb40 2020-05-18T21:40:18.327 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3ca59c700 1 --2- 172.21.15.80:0/118865453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 secure :-1 s=THROTTLE_DONE pgs=27 cs=0 l=1 rx=0x7fa3c0002f70 tx=0x7fa3c0010ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.327 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3ca59c700 1 --2- 172.21.15.80:0/118865453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 secure :-1 s=THROTTLE_DONE pgs=27 cs=0 l=1 rx=0x7fa3c0002f70 tx=0x7fa3c0010ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.327 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3bb7fe700 1 -- 172.21.15.80:0/118865453 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c00072e0 con 0x7fa3cc0abb40 2020-05-18T21:40:18.327 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 -- 172.21.15.80:0/118865453 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc067200 con 0x7fa3cc0abb40 2020-05-18T21:40:18.327 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3bb7fe700 1 -- 172.21.15.80:0/118865453 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3c0012340 con 0x7fa3cc0abb40 2020-05-18T21:40:18.327 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3cb59e700 1 -- 172.21.15.80:0/118865453 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc130150 con 0x7fa3cc0abb40 2020-05-18T21:40:18.328 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3ca59c700 1 --2- 172.21.15.80:0/118865453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 secure :-1 s=THROTTLE_DONE pgs=27 cs=0 l=1 rx=0x7fa3c0002f70 tx=0x7fa3c0010ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.328 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3bb7fe700 1 -- 172.21.15.80:0/118865453 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) v1 ==== 514+0+0 (secure 0 0 0) 0x7fa3c0012ab0 con 0x7fa3cc0abb40 2020-05-18T21:40:18.328 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3ca59c700 1 --2- 172.21.15.80:0/118865453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 secure :-1 s=THROTTLE_DONE pgs=27 cs=0 l=1 rx=0x7fa3c0002f70 tx=0x7fa3c0010ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.328 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.305+0000 7fa3bb7fe700 1 -- 172.21.15.80:0/118865453 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7fa3c000e2d0 con 0x7fa3cc0abb40 2020-05-18T21:40:18.328 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.309+0000 7fa3cb59e700 1 -- 172.21.15.80:0/118865453 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3cc0569e0 con 0x7fa3cc0abb40 2020-05-18T21:40:18.329 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.309+0000 7fa3ca59c700 1 --2- 172.21.15.80:0/118865453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 secure :-1 s=THROTTLE_DONE pgs=27 cs=0 l=1 rx=0x7fa3c0002f70 tx=0x7fa3c0010ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.329 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.309+0000 7fa3bb7fe700 1 -- 172.21.15.80:0/118865453 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67708 (secure 0 0 0) 0x7fa3c0012d40 con 0x7fa3cc0abb40 2020-05-18T21:40:18.382 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.377+0000 7fa3cb59e700 1 -- 172.21.15.80:0/118865453 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e42df2e8-9cc5-4c7e-a243-bd18ccfbd002", "id": 6} v 0) v1 -- 0x7fa3cc14f130 con 0x7fa3cc0abb40 2020-05-18T21:40:18.390 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.385+0000 7fa3ca59c700 1 --2- 172.21.15.80:0/118865453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 secure :-1 s=THROTTLE_DONE pgs=27 cs=0 l=1 rx=0x7fa3c0002f70 tx=0x7fa3c0010ca0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.391 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.385+0000 7fa3bb7fe700 1 -- 172.21.15.80:0/118865453 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e42df2e8-9cc5-4c7e-a243-bd18ccfbd002", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa3c000e730 con 0x7fa3cc0abb40 2020-05-18T21:40:18.391 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.385+0000 7fa3cb59e700 1 -- 172.21.15.80:0/118865453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc0abb40 msgr2=0x7fa3cc14e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.391 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.385+0000 7fa3cb59e700 1 --2- 172.21.15.80:0/118865453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 secure :-1 s=READY pgs=27 cs=0 l=1 rx=0x7fa3c0002f70 tx=0x7fa3c0010ca0).stop 2020-05-18T21:40:18.392 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.385+0000 7fa3cb59e700 1 -- 172.21.15.80:0/118865453 shutdown_connections 2020-05-18T21:40:18.392 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.385+0000 7fa3cb59e700 1 --2- 172.21.15.80:0/118865453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3cc0abb40 0x7fa3cc14e1b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:18.392 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.385+0000 7fa3cb59e700 1 -- 172.21.15.80:0/118865453 >> 172.21.15.80:0/118865453 conn(0x7fa3cc069ac0 msgr2=0x7fa3cc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:18.393 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.385+0000 7fa3cb59e700 1 -- 172.21.15.80:0/118865453 shutdown_connections 2020-05-18T21:40:18.393 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.385+0000 7fa3cb59e700 1 -- 172.21.15.80:0/118865453 wait complete. 2020-05-18T21:40:18.393 INFO:teuthology.orchestra.run.smithi080.stdout:6 2020-05-18T21:40:18.404 INFO:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new ebd6c938-beff-4668-b70e-076ed22bb8c3 7 2020-05-18T21:40:18.506 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.497+0000 7f7135bf8700 1 Processor -- start 2020-05-18T21:40:18.507 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.497+0000 7f7135bf8700 1 -- start start 2020-05-18T21:40:18.507 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f7135bf8700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f71300a69d0 0x7f71300a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:18.507 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f7135bf8700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71300a3eb0 con 0x7f71300a7070 2020-05-18T21:40:18.507 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f7135bf8700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71300a4150 con 0x7f71300a4d70 2020-05-18T21:40:18.508 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f7135bf8700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71300a5f00 con 0x7f71300a69d0 2020-05-18T21:40:18.508 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712ffff700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f71300a7070 0x7f71300a62c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35386/0 (socket says 172.21.15.80:35386) 2020-05-18T21:40:18.508 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712ffff700 1 -- 172.21.15.80:0/804652963 learned_addr learned my addr 172.21.15.80:0/804652963 (peer_addr_for_me v1:172.21.15.80:0/0) 2020-05-18T21:40:18.508 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712f7fe700 1 --2- 172.21.15.80:0/804652963 >> v2:172.21.15.196:3300/0 conn(0x7f71300a69d0 0x7f71300a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:18.509 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712effd700 1 -- 172.21.15.80:0/804652963 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946942013 0 0) 0x7f71300a3eb0 con 0x7f71300a7070 2020-05-18T21:40:18.509 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712effd700 1 -- 172.21.15.80:0/804652963 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f711c003270 con 0x7f71300a7070 2020-05-18T21:40:18.509 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712f7fe700 1 -- 172.21.15.80:0/804652963 >> v1:172.21.15.196:6789/0 conn(0x7f71300a4d70 legacy=0x7f713014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:18.509 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712f7fe700 1 -- 172.21.15.80:0/804652963 >> v1:172.21.15.80:6789/0 conn(0x7f71300a7070 legacy=0x7f71300a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.510 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712f7fe700 1 -- 172.21.15.80:0/804652963 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71300a3be0 con 0x7f71300a69d0 2020-05-18T21:40:18.510 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712f7fe700 1 --2- 172.21.15.80:0/804652963 >> v2:172.21.15.196:3300/0 conn(0x7f71300a69d0 0x7f71300a45f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.510 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712f7fe700 1 --2- 172.21.15.80:0/804652963 >> v2:172.21.15.196:3300/0 conn(0x7f71300a69d0 0x7f71300a45f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.510 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712f7fe700 1 --2- 172.21.15.80:0/804652963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a45f0 secure :-1 s=READY pgs=27 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).ready entity=mon.1 client_cookie=5a55d32087da0c27 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:18.510 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712f7fe700 1 --2- 172.21.15.80:0/804652963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a45f0 secure :-1 s=THROTTLE_DONE pgs=27 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.511 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712effd700 1 -- 172.21.15.80:0/804652963 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7118008e80 con 0x7f71300a69d0 2020-05-18T21:40:18.511 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712f7fe700 1 --2- 172.21.15.80:0/804652963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a45f0 secure :-1 s=THROTTLE_DONE pgs=27 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.511 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712effd700 1 -- 172.21.15.80:0/804652963 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71180115b0 con 0x7f71300a69d0 2020-05-18T21:40:18.511 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712f7fe700 1 --2- 172.21.15.80:0/804652963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a45f0 secure :-1 s=THROTTLE_DONE pgs=27 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.511 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f7135bf8700 1 -- 172.21.15.80:0/804652963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 msgr2=0x7f71300a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.512 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f7135bf8700 1 --2- 172.21.15.80:0/804652963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a45f0 secure :-1 s=READY pgs=27 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).stop 2020-05-18T21:40:18.512 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f712effd700 1 -- 172.21.15.80:0/804652963 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7118011990 con 0x7f71300a69d0 2020-05-18T21:40:18.512 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f7135bf8700 1 -- 172.21.15.80:0/804652963 shutdown_connections 2020-05-18T21:40:18.512 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f7135bf8700 1 --2- 172.21.15.80:0/804652963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a45f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:18.512 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f7135bf8700 1 -- 172.21.15.80:0/804652963 >> 172.21.15.80:0/804652963 conn(0x7f713005a7c0 msgr2=0x7f713005aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:18.513 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f7135bf8700 1 -- 172.21.15.80:0/804652963 shutdown_connections 2020-05-18T21:40:18.513 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f7135bf8700 1 -- 172.21.15.80:0/804652963 wait complete. 2020-05-18T21:40:18.513 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f7135bf8700 1 Processor -- start 2020-05-18T21:40:18.513 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.501+0000 7f7135bf8700 1 -- start start 2020-05-18T21:40:18.516 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f7135bf8700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f71300a69d0 0x7f71300a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:18.517 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f7135bf8700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f71300a7070 0x7f713009fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:18.517 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f7135bf8700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7130123010 con 0x7f71300a4d70 2020-05-18T21:40:18.517 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f7135bf8700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71300a5340 con 0x7f71300a7070 2020-05-18T21:40:18.517 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f7135bf8700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71301232b0 con 0x7f71300a69d0 2020-05-18T21:40:18.518 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f7134996700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f71300a7070 0x7f713009fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:18.518 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f7134996700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f71300a7070 0x7f713009fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:33284/0 (socket says 172.21.15.80:33284) 2020-05-18T21:40:18.518 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f7134996700 1 -- 172.21.15.80:0/758250000 learned_addr learned my addr 172.21.15.80:0/758250000 (peer_addr_for_me v2:172.21.15.80:0/0) 2020-05-18T21:40:18.518 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712f7fe700 1 --2- 172.21.15.80:0/758250000 >> v2:172.21.15.196:3300/0 conn(0x7f71300a69d0 0x7f71300a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:18.518 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712f7fe700 1 -- 172.21.15.80:0/758250000 >> v2:172.21.15.80:3300/0 conn(0x7f71300a7070 msgr2=0x7f713009fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712f7fe700 1 --2- 172.21.15.80:0/758250000 >> v2:172.21.15.80:3300/0 conn(0x7f71300a7070 0x7f713009fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712f7fe700 1 -- 172.21.15.80:0/758250000 >> v1:172.21.15.196:6789/0 conn(0x7f71300a4d70 legacy=0x7f713009d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712f7fe700 1 -- 172.21.15.80:0/758250000 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7130123550 con 0x7f71300a69d0 2020-05-18T21:40:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f7134996700 1 --2- 172.21.15.80:0/758250000 >> v2:172.21.15.80:3300/0 conn(0x7f71300a7070 0x7f713009fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712f7fe700 1 --2- 172.21.15.80:0/758250000 >> v2:172.21.15.196:3300/0 conn(0x7f71300a69d0 0x7f71300a6390 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.520 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712f7fe700 1 --2- 172.21.15.80:0/758250000 >> v2:172.21.15.196:3300/0 conn(0x7f71300a69d0 0x7f71300a6390 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.520 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712f7fe700 1 --2- 172.21.15.80:0/758250000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a6390 secure :-1 s=READY pgs=29 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:18.520 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712f7fe700 1 --2- 172.21.15.80:0/758250000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a6390 secure :-1 s=THROTTLE_DONE pgs=29 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.520 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712cff9700 1 -- 172.21.15.80:0/758250000 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71180119e0 con 0x7f71300a69d0 2020-05-18T21:40:18.520 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712f7fe700 1 --2- 172.21.15.80:0/758250000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a6390 secure :-1 s=THROTTLE_DONE pgs=29 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.521 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712cff9700 1 -- 172.21.15.80:0/758250000 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7118007360 con 0x7f71300a69d0 2020-05-18T21:40:18.521 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f7135bf8700 1 -- 172.21.15.80:0/758250000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71301237f0 con 0x7f71300a69d0 2020-05-18T21:40:18.521 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712f7fe700 1 --2- 172.21.15.80:0/758250000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a6390 secure :-1 s=THROTTLE_DONE pgs=29 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.521 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712cff9700 1 -- 172.21.15.80:0/758250000 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7118005700 con 0x7f71300a69d0 2020-05-18T21:40:18.521 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f7135bf8700 1 -- 172.21.15.80:0/758250000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713009d970 con 0x7f71300a69d0 2020-05-18T21:40:18.522 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712f7fe700 1 --2- 172.21.15.80:0/758250000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a6390 secure :-1 s=THROTTLE_DONE pgs=29 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.522 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712cff9700 1 -- 172.21.15.80:0/758250000 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) v1 ==== 514+0+0 (secure 0 0 0) 0x7f711800b480 con 0x7f71300a69d0 2020-05-18T21:40:18.522 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712f7fe700 1 --2- 172.21.15.80:0/758250000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a6390 secure :-1 s=THROTTLE_DONE pgs=29 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.522 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f712cff9700 1 -- 172.21.15.80:0/758250000 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7f7118005e40 con 0x7f71300a69d0 2020-05-18T21:40:18.522 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.505+0000 7f7135bf8700 1 -- 172.21.15.80:0/758250000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7130123980 con 0x7f71300a69d0 2020-05-18T21:40:18.523 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.513+0000 7f712f7fe700 1 --2- 172.21.15.80:0/758250000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a6390 secure :-1 s=THROTTLE_DONE pgs=29 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.523 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.513+0000 7f712cff9700 1 -- 172.21.15.80:0/758250000 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67708 (secure 0 0 0) 0x7f711800ba80 con 0x7f71300a69d0 2020-05-18T21:40:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.581+0000 7f7135bf8700 1 -- 172.21.15.80:0/758250000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ebd6c938-beff-4668-b70e-076ed22bb8c3", "id": 7} v 0) v1 -- 0x7f71300569e0 con 0x7f71300a69d0 2020-05-18T21:40:18.595 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.585+0000 7f712f7fe700 1 --2- 172.21.15.80:0/758250000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a6390 secure :-1 s=THROTTLE_DONE pgs=29 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:18.595 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.585+0000 7f712cff9700 1 -- 172.21.15.80:0/758250000 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ebd6c938-beff-4668-b70e-076ed22bb8c3", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f711800bc20 con 0x7f71300a69d0 2020-05-18T21:40:18.596 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.589+0000 7f7135bf8700 1 -- 172.21.15.80:0/758250000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 msgr2=0x7f71300a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:18.596 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.589+0000 7f7135bf8700 1 --2- 172.21.15.80:0/758250000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a6390 secure :-1 s=READY pgs=29 cs=0 l=1 rx=0x7f713005b610 tx=0x7f7118007a80).stop 2020-05-18T21:40:18.596 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.589+0000 7f7135bf8700 1 -- 172.21.15.80:0/758250000 shutdown_connections 2020-05-18T21:40:18.596 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.589+0000 7f7135bf8700 1 --2- 172.21.15.80:0/758250000 >> v2:172.21.15.80:3300/0 conn(0x7f71300a7070 0x7f713009fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:18.597 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.589+0000 7f7135bf8700 1 --2- 172.21.15.80:0/758250000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71300a69d0 0x7f71300a6390 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:18.597 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.589+0000 7f7135bf8700 1 -- 172.21.15.80:0/758250000 >> 172.21.15.80:0/758250000 conn(0x7f713005a7c0 msgr2=0x7f713007f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:18.597 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.589+0000 7f7135bf8700 1 -- 172.21.15.80:0/758250000 shutdown_connections 2020-05-18T21:40:18.598 INFO:teuthology.orchestra.run.smithi080.stderr:2020-05-18T21:40:18.589+0000 7f7135bf8700 1 -- 172.21.15.80:0/758250000 wait complete. 2020-05-18T21:40:18.598 INFO:teuthology.orchestra.run.smithi080.stdout:7 2020-05-18T21:40:18.602 INFO:tasks.ceph.osd.0:Restarting daemon 2020-05-18T21:40:18.602 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-05-18T21:40:18.606 INFO:tasks.ceph.osd.0:Started 2020-05-18T21:40:18.606 INFO:tasks.ceph.osd.1:Restarting daemon 2020-05-18T21:40:18.607 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-05-18T21:40:18.609 INFO:tasks.ceph.osd.1:Started 2020-05-18T21:40:18.610 INFO:tasks.ceph.osd.2:Restarting daemon 2020-05-18T21:40:18.610 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-05-18T21:40:18.612 INFO:tasks.ceph.osd.2:Started 2020-05-18T21:40:18.613 INFO:tasks.ceph.osd.3:Restarting daemon 2020-05-18T21:40:18.613 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2020-05-18T21:40:18.616 INFO:tasks.ceph.osd.3:Started 2020-05-18T21:40:18.616 INFO:tasks.ceph.osd.4:Restarting daemon 2020-05-18T21:40:18.617 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2020-05-18T21:40:18.619 INFO:tasks.ceph.osd.4:Started 2020-05-18T21:40:18.619 INFO:tasks.ceph.osd.5:Restarting daemon 2020-05-18T21:40:18.620 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2020-05-18T21:40:18.622 INFO:tasks.ceph.osd.5:Started 2020-05-18T21:40:18.622 INFO:tasks.ceph.osd.6:Restarting daemon 2020-05-18T21:40:18.623 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2020-05-18T21:40:18.625 INFO:tasks.ceph.osd.6:Started 2020-05-18T21:40:18.625 INFO:tasks.ceph.osd.7:Restarting daemon 2020-05-18T21:40:18.626 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2020-05-18T21:40:18.629 INFO:tasks.ceph.osd.7:Started 2020-05-18T21:40:18.629 INFO:tasks.ceph:Waiting for OSDs to come up 2020-05-18T21:40:18.967 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:40:18.967+0000 7f73e4ff1dc0 -1 Falling back to public interface 2020-05-18T21:40:19.235 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:40:19.235+0000 7fe8a7087dc0 -1 Falling back to public interface 2020-05-18T21:40:19.243 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:40:19.243+0000 7fea90044dc0 -1 Falling back to public interface 2020-05-18T21:40:19.245 INFO:tasks.ceph.osd.7.smithi080.stderr:2020-05-18T21:40:19.241+0000 7fc69cea8dc0 -1 Falling back to public interface 2020-05-18T21:40:19.245 INFO:tasks.ceph.osd.6.smithi080.stderr:2020-05-18T21:40:19.241+0000 7f2fb7b6fdc0 -1 Falling back to public interface 2020-05-18T21:40:19.245 INFO:tasks.ceph.osd.6.smithi080.stderr:2020-05-18T21:40:19.241+0000 7f2fb7336700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2020-05-18T21:40:19.249 INFO:tasks.ceph.osd.4.smithi080.stderr:2020-05-18T21:40:19.245+0000 7ff223d5cdc0 -1 Falling back to public interface 2020-05-18T21:40:19.249 INFO:tasks.ceph.osd.5.smithi080.stderr:2020-05-18T21:40:19.245+0000 7f0878eacdc0 -1 Falling back to public interface 2020-05-18T21:40:19.254 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:40:19.255+0000 7f134f0f2dc0 -1 Falling back to public interface 2020-05-18T21:40:19.437 INFO:teuthology.orchestra.run.smithi196:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-05-18T21:40:19.529 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.523+0000 7f7faafcf700 1 Processor -- start 2020-05-18T21:40:19.529 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.523+0000 7f7faafcf700 1 -- start start 2020-05-18T21:40:19.530 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faafcf700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7fa40a68d0 0x7f7fa40a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:19.530 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faafcf700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa4024770 con 0x7f7fa4024ba0 2020-05-18T21:40:19.530 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faafcf700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa40a9840 con 0x7f7fa405f260 2020-05-18T21:40:19.530 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faafcf700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa4025050 con 0x7f7fa40a68d0 2020-05-18T21:40:19.531 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7fac231700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f7fa405f260 0x7f7fa414f710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42574/0 (socket says 172.21.15.196:42574) 2020-05-18T21:40:19.531 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7fac231700 1 -- 172.21.15.196:0/3151645213 learned_addr learned my addr 172.21.15.196:0/3151645213 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:19.531 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/3151645213 >> v2:172.21.15.80:3300/0 conn(0x7f7fa40a68d0 0x7f7fa40a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:19.531 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7fa97cc700 1 -- 172.21.15.196:0/3151645213 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442267520 0 0) 0x7f7fa40a9840 con 0x7f7fa405f260 2020-05-18T21:40:19.531 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7fa97cc700 1 -- 172.21.15.196:0/3151645213 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f90003270 con 0x7f7fa405f260 2020-05-18T21:40:19.532 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faa7ce700 1 -- 172.21.15.196:0/3151645213 >> v1:172.21.15.196:6789/0 conn(0x7f7fa405f260 legacy=0x7f7fa414f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:19.532 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faa7ce700 1 -- 172.21.15.196:0/3151645213 >> v1:172.21.15.80:6789/0 conn(0x7f7fa4024ba0 legacy=0x7f7fa405eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:19.532 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faa7ce700 1 -- 172.21.15.196:0/3151645213 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa4061450 con 0x7f7fa40a68d0 2020-05-18T21:40:19.532 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/3151645213 >> v2:172.21.15.80:3300/0 conn(0x7f7fa40a68d0 0x7f7fa40a6f60 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f7fa0002f70 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.533 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/3151645213 >> v2:172.21.15.80:3300/0 conn(0x7f7fa40a68d0 0x7f7fa40a6f60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f7fa0002f70 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.533 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/3151645213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7fa40a68d0 0x7f7fa40a6f60 secure :-1 s=READY pgs=37 cs=0 l=1 rx=0x7f7fa0002f70 tx=0x7f7fa0007a80).ready entity=mon.0 client_cookie=4bc2983afa27865d server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:19.533 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/3151645213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7fa40a68d0 0x7f7fa40a6f60 secure :-1 s=THROTTLE_DONE pgs=37 cs=0 l=1 rx=0x7f7fa0002f70 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.533 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7fa97cc700 1 -- 172.21.15.196:0/3151645213 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa0008de0 con 0x7f7fa40a68d0 2020-05-18T21:40:19.534 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/3151645213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7fa40a68d0 0x7f7fa40a6f60 secure :-1 s=THROTTLE_DONE pgs=37 cs=0 l=1 rx=0x7f7fa0002f70 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.534 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7fa97cc700 1 -- 172.21.15.196:0/3151645213 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa0011790 con 0x7f7fa40a68d0 2020-05-18T21:40:19.534 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/3151645213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7fa40a68d0 0x7f7fa40a6f60 secure :-1 s=THROTTLE_DONE pgs=37 cs=0 l=1 rx=0x7f7fa0002f70 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.534 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7fa97cc700 1 -- 172.21.15.196:0/3151645213 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa0011af0 con 0x7f7fa40a68d0 2020-05-18T21:40:19.535 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faafcf700 1 -- 172.21.15.196:0/3151645213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7fa40a68d0 msgr2=0x7f7fa40a6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:19.535 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faafcf700 1 --2- 172.21.15.196:0/3151645213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7fa40a68d0 0x7f7fa40a6f60 secure :-1 s=READY pgs=37 cs=0 l=1 rx=0x7f7fa0002f70 tx=0x7f7fa0007a80).stop 2020-05-18T21:40:19.535 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faafcf700 1 -- 172.21.15.196:0/3151645213 shutdown_connections 2020-05-18T21:40:19.535 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faafcf700 1 --2- 172.21.15.196:0/3151645213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7fa40a68d0 0x7f7fa40a6f60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:19.536 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faafcf700 1 -- 172.21.15.196:0/3151645213 >> 172.21.15.196:0/3151645213 conn(0x7f7fa40a8460 msgr2=0x7f7fa40a8870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:19.536 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faafcf700 1 -- 172.21.15.196:0/3151645213 shutdown_connections 2020-05-18T21:40:19.536 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faafcf700 1 -- 172.21.15.196:0/3151645213 wait complete. 2020-05-18T21:40:19.536 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faafcf700 1 Processor -- start 2020-05-18T21:40:19.536 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.527+0000 7f7faafcf700 1 -- start start 2020-05-18T21:40:19.545 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faafcf700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7fa40a68d0 0x7f7fa412a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:19.546 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faafcf700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa4070ec0 con 0x7f7fa4024ba0 2020-05-18T21:40:19.546 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faafcf700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa4071160 con 0x7f7fa405f260 2020-05-18T21:40:19.546 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faafcf700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa4070cd0 con 0x7f7fa40a68d0 2020-05-18T21:40:19.546 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7fa9fcd700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f7fa405f260 0x7f7fa4127c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42576/0 (socket says 172.21.15.196:42576) 2020-05-18T21:40:19.551 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7fa9fcd700 1 -- 172.21.15.196:0/2795251864 learned_addr learned my addr 172.21.15.196:0/2795251864 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:19.552 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7fac231700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7fa40a68d0 0x7f7fa412a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:19.552 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752818467 0 0) 0x7f7fa4071160 con 0x7f7fa405f260 2020-05-18T21:40:19.552 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f8c003270 con 0x7f7fa405f260 2020-05-18T21:40:19.552 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216513919 0 0) 0x7f7fa4070ec0 con 0x7f7fa4024ba0 2020-05-18T21:40:19.552 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fa4071160 con 0x7f7fa4024ba0 2020-05-18T21:40:19.553 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1406611859 0 0) 0x7f7f8c003270 con 0x7f7fa405f260 2020-05-18T21:40:19.553 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 --> v1:172.21.15.196:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7fa4070ec0 con 0x7f7fa405f260 2020-05-18T21:40:19.553 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7f7f94001970 con 0x7f7fa405f260 2020-05-18T21:40:19.553 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7fa40a68d0 msgr2=0x7f7fa412a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:19.554 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7fa40a68d0 0x7f7fa412a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:19.555 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 >> v1:172.21.15.196:6789/0 conn(0x7f7fa405f260 legacy=0x7f7fa4127c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:19.555 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 >> v1:172.21.15.80:6789/0 conn(0x7f7fa4024ba0 legacy=0x7f7fa4137940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:19.556 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 0x7f7f8c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:19.556 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f8c004510 0x7f7f8c004970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:19.556 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7fa40a68d0 0x7f7f8c004dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:19.557 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c005230 con 0x7f7f8c004510 2020-05-18T21:40:19.557 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0053a0 con 0x7f7f8c0039a0 2020-05-18T21:40:19.557 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c00b760 con 0x7f7fa40a68d0 2020-05-18T21:40:19.558 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 0x7f7f8c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:19.558 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7fa9fcd700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f8c004510 0x7f7f8c004970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:19.558 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7fac231700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7fa40a68d0 0x7f7f8c004dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:19.558 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 -- 172.21.15.196:0/2795251864 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7fa40a68d0 msgr2=0x7f7f8c004dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:19.559 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7fa40a68d0 0x7f7f8c004dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:19.559 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 -- 172.21.15.196:0/2795251864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f8c004510 msgr2=0x7f7f8c004970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:19.559 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f8c004510 0x7f7f8c004970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:19.559 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 -- 172.21.15.196:0/2795251864 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa40a4df0 con 0x7f7f8c0039a0 2020-05-18T21:40:19.559 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faafcf700 1 -- 172.21.15.196:0/2795251864 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa4138050 con 0x7f7f8c0039a0 2020-05-18T21:40:19.560 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 0x7f7f8c003dc0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7fa0008920 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.560 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faafcf700 1 -- 172.21.15.196:0/2795251864 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa406f9d0 con 0x7f7f8c0039a0 2020-05-18T21:40:19.560 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 0x7f7f8c003dc0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7fa0008920 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.560 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 0x7f7f8c003dc0 secure :-1 s=READY pgs=41 cs=0 l=1 rx=0x7f7fa0008920 tx=0x7f7fa0007a80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:19.560 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 0x7f7f8c003dc0 secure :-1 s=THROTTLE_DONE pgs=41 cs=0 l=1 rx=0x7f7fa0008920 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.561 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 0x7f7f8c003dc0 secure :-1 s=THROTTLE_DONE pgs=41 cs=0 l=1 rx=0x7f7fa0008920 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.561 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa00078c0 con 0x7f7f8c0039a0 2020-05-18T21:40:19.561 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 0x7f7f8c003dc0 secure :-1 s=THROTTLE_DONE pgs=41 cs=0 l=1 rx=0x7f7fa0008920 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.561 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 0x7f7f8c003dc0 secure :-1 s=THROTTLE_DONE pgs=41 cs=0 l=1 rx=0x7f7fa0008920 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.561 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa0007360 con 0x7f7f8c0039a0 2020-05-18T21:40:19.562 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa0005700 con 0x7f7f8c0039a0 2020-05-18T21:40:19.562 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 0x7f7f8c003dc0 secure :-1 s=THROTTLE_DONE pgs=41 cs=0 l=1 rx=0x7f7fa0008920 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.562 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) v1 ==== 514+0+0 (secure 0 0 0) 0x7f7fa0012440 con 0x7f7f8c0039a0 2020-05-18T21:40:19.562 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f7fa0005e00 con 0x7f7f8c0039a0 2020-05-18T21:40:19.563 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.531+0000 7f7faafcf700 1 -- 172.21.15.196:0/2795251864 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f80003380 con 0x7f7f8c0039a0 2020-05-18T21:40:19.563 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.535+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 0x7f7f8c003dc0 secure :-1 s=THROTTLE_DONE pgs=41 cs=0 l=1 rx=0x7f7fa0008920 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.563 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.535+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67708 (secure 0 0 0) 0x7f7fa000b0d0 con 0x7f7f8c0039a0 2020-05-18T21:40:19.594 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.591+0000 7f7faafcf700 1 -- 172.21.15.196:0/2795251864 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f80003730 con 0x7f7f8c0039a0 2020-05-18T21:40:19.595 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.595+0000 7f7faa7ce700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 0x7f7f8c003dc0 secure :-1 s=THROTTLE_DONE pgs=41 cs=0 l=1 rx=0x7f7fa0008920 tx=0x7f7fa0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:19.595 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.595+0000 7f7f9affd700 1 -- 172.21.15.196:0/2795251864 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+6852 (secure 0 0 0) 0x7f7fa0012900 con 0x7f7f8c0039a0 2020-05-18T21:40:19.595 INFO:teuthology.misc.health.smithi196.stdout: 2020-05-18T21:40:19.595 INFO:teuthology.misc.health.smithi196.stdout:{"epoch":10,"fsid":"d048673a-3d91-4cc6-882d-e9cc70e7cd6c","created":"2020-05-18T21:40:15.911738+0000","modified":"2020-05-18T21:40:18.586161+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"8db63a0a-61dc-42b5-9559-fddcf436cab5","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6773b0c3-9f49-42a2-b75b-c10c734ff086","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ac1f2e05-9bb7-4768-b712-10e4d2ea4154","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bae5ab40-235b-4e0f-8d0d-524afd099b1a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"275fc52c-58e4-4f35-a0cb-e117fe36bf5c","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9795b89c-e2f8-4abf-b9c7-796727a57002","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e42df2e8-9cc5-4c7e-a243-bd18ccfbd002","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ebd6c938-beff-4668-b70e-076ed22bb8c3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-18T21:40:19.612 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.607+0000 7f7faafcf700 1 -- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 msgr2=0x7f7f8c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:19.613 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.607+0000 7f7faafcf700 1 --2- 172.21.15.196:0/2795251864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f8c0039a0 0x7f7f8c003dc0 secure :-1 s=READY pgs=41 cs=0 l=1 rx=0x7f7fa0008920 tx=0x7f7fa0007a80).stop 2020-05-18T21:40:19.613 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.607+0000 7f7fac231700 1 -- 172.21.15.196:0/2795251864 reap_dead start 2020-05-18T21:40:19.613 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.607+0000 7f7faafcf700 1 -- 172.21.15.196:0/2795251864 shutdown_connections 2020-05-18T21:40:19.613 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.607+0000 7f7faafcf700 1 -- 172.21.15.196:0/2795251864 >> 172.21.15.196:0/2795251864 conn(0x7f7fa40a8460 msgr2=0x7f7fa405bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:19.614 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.607+0000 7f7faafcf700 1 -- 172.21.15.196:0/2795251864 shutdown_connections 2020-05-18T21:40:19.614 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:19.607+0000 7f7faafcf700 1 -- 172.21.15.196:0/2795251864 wait complete. 2020-05-18T21:40:19.616 DEBUG:teuthology.misc:0 of 8 OSDs are up 2020-05-18T21:40:20.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:40:20.151+0000 7f73e4ff1dc0 -1 osd.0 0 log_to_monitors {default=true} 2020-05-18T21:40:20.194 INFO:tasks.ceph.osd.7.smithi080.stderr:2020-05-18T21:40:20.189+0000 7fc69cea8dc0 -1 osd.7 0 log_to_monitors {default=true} 2020-05-18T21:40:20.204 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:40:20.203+0000 7f134f0f2dc0 -1 osd.3 0 log_to_monitors {default=true} 2020-05-18T21:40:20.210 INFO:tasks.ceph.osd.6.smithi080.stderr:2020-05-18T21:40:20.209+0000 7f2fb7b6fdc0 -1 osd.6 0 log_to_monitors {default=true} 2020-05-18T21:40:20.406 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:40:20.403+0000 7fea90044dc0 -1 osd.1 0 log_to_monitors {default=true} 2020-05-18T21:40:20.421 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:40:20.419+0000 7fe8a7087dc0 -1 osd.2 0 log_to_monitors {default=true} 2020-05-18T21:40:20.429 INFO:tasks.ceph.osd.4.smithi080.stderr:2020-05-18T21:40:20.425+0000 7ff223d5cdc0 -1 osd.4 0 log_to_monitors {default=true} 2020-05-18T21:40:20.443 INFO:tasks.ceph.osd.5.smithi080.stderr:2020-05-18T21:40:20.441+0000 7f0878eacdc0 -1 osd.5 0 log_to_monitors {default=true} 2020-05-18T21:40:21.776 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:40:21.775+0000 7f1341132700 -1 osd.3 0 waiting for initial osdmap 2020-05-18T21:40:21.777 INFO:tasks.ceph.osd.4.smithi080.stderr:2020-05-18T21:40:21.773+0000 7ff215d9c700 -1 osd.4 0 waiting for initial osdmap 2020-05-18T21:40:21.777 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:40:21.775+0000 7fe8990c7700 -1 osd.2 0 waiting for initial osdmap 2020-05-18T21:40:21.778 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:40:21.775+0000 7fea82084700 -1 osd.1 0 waiting for initial osdmap 2020-05-18T21:40:21.778 INFO:tasks.ceph.osd.5.smithi080.stderr:2020-05-18T21:40:21.773+0000 7f086aeec700 -1 osd.5 0 waiting for initial osdmap 2020-05-18T21:40:21.779 INFO:tasks.ceph.osd.6.smithi080.stderr:2020-05-18T21:40:21.777+0000 7f2fa9baf700 -1 osd.6 0 waiting for initial osdmap 2020-05-18T21:40:21.779 INFO:tasks.ceph.osd.7.smithi080.stderr:2020-05-18T21:40:21.777+0000 7fc68eee8700 -1 osd.7 0 waiting for initial osdmap 2020-05-18T21:40:21.780 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:40:21.779+0000 7f73d7031700 -1 osd.0 0 waiting for initial osdmap 2020-05-18T21:40:26.423 INFO:teuthology.orchestra.run.smithi196:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-05-18T21:40:26.527 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.519+0000 7fc34ffff700 1 Processor -- start 2020-05-18T21:40:26.528 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.519+0000 7fc34ffff700 1 -- start start 2020-05-18T21:40:26.528 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34ffff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc3500a68d0 0x7fc3500a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:26.528 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34ffff700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc350024770 con 0x7fc35005f260 2020-05-18T21:40:26.528 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34ffff700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3500a9840 con 0x7fc350024ba0 2020-05-18T21:40:26.529 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34ffff700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc350025050 con 0x7fc3500a68d0 2020-05-18T21:40:26.529 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34f7fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc3500a68d0 0x7fc3500a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:26.529 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34f7fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc3500a68d0 0x7fc3500a6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34098/0 (socket says 172.21.15.196:34098) 2020-05-18T21:40:26.529 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc347fff700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc350024ba0 0x7fc35005eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42860/0 (socket says 172.21.15.196:42860) 2020-05-18T21:40:26.529 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34f7fe700 1 -- 172.21.15.196:0/347170304 learned_addr learned my addr 172.21.15.196:0/347170304 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:26.530 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34f7fe700 1 -- 172.21.15.196:0/347170304 >> v1:172.21.15.196:6789/0 conn(0x7fc350024ba0 legacy=0x7fc35005eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:26.530 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34f7fe700 1 -- 172.21.15.196:0/347170304 >> v1:172.21.15.80:6789/0 conn(0x7fc35005f260 legacy=0x7fc35014f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:26.530 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34f7fe700 1 -- 172.21.15.196:0/347170304 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc350061450 con 0x7fc3500a68d0 2020-05-18T21:40:26.530 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34effd700 1 -- 172.21.15.196:0/347170304 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535749131 0 0) 0x7fc3500a9840 con 0x7fc350024ba0 2020-05-18T21:40:26.531 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34f7fe700 1 --2- 172.21.15.196:0/347170304 >> v2:172.21.15.196:3300/0 conn(0x7fc3500a68d0 0x7fc3500a6f60 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fc340002f70 tx=0x7fc340007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.531 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34f7fe700 1 --2- 172.21.15.196:0/347170304 >> v2:172.21.15.196:3300/0 conn(0x7fc3500a68d0 0x7fc3500a6f60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fc340002f70 tx=0x7fc340007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.531 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34f7fe700 1 --2- 172.21.15.196:0/347170304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3500a68d0 0x7fc3500a6f60 secure :-1 s=READY pgs=61 cs=0 l=1 rx=0x7fc340002f70 tx=0x7fc340007a80).ready entity=mon.1 client_cookie=eb78e3406d3f75d1 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:26.531 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34f7fe700 1 --2- 172.21.15.196:0/347170304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3500a68d0 0x7fc3500a6f60 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7fc340002f70 tx=0x7fc340007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.531 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34f7fe700 1 --2- 172.21.15.196:0/347170304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3500a68d0 0x7fc3500a6f60 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7fc340002f70 tx=0x7fc340007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.532 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34effd700 1 -- 172.21.15.196:0/347170304 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc340008e50 con 0x7fc3500a68d0 2020-05-18T21:40:26.532 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34f7fe700 1 --2- 172.21.15.196:0/347170304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3500a68d0 0x7fc3500a6f60 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7fc340002f70 tx=0x7fc340007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.533 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34effd700 1 -- 172.21.15.196:0/347170304 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc340010800 con 0x7fc3500a68d0 2020-05-18T21:40:26.533 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34effd700 1 -- 172.21.15.196:0/347170304 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc340010ba0 con 0x7fc3500a68d0 2020-05-18T21:40:26.534 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34ffff700 1 -- 172.21.15.196:0/347170304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3500a68d0 msgr2=0x7fc3500a6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:26.534 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34ffff700 1 --2- 172.21.15.196:0/347170304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3500a68d0 0x7fc3500a6f60 secure :-1 s=READY pgs=61 cs=0 l=1 rx=0x7fc340002f70 tx=0x7fc340007a80).stop 2020-05-18T21:40:26.534 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34ffff700 1 -- 172.21.15.196:0/347170304 shutdown_connections 2020-05-18T21:40:26.534 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34ffff700 1 --2- 172.21.15.196:0/347170304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3500a68d0 0x7fc3500a6f60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:26.534 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34ffff700 1 -- 172.21.15.196:0/347170304 >> 172.21.15.196:0/347170304 conn(0x7fc3500a8460 msgr2=0x7fc3500a8870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:26.535 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34ffff700 1 -- 172.21.15.196:0/347170304 shutdown_connections 2020-05-18T21:40:26.535 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.523+0000 7fc34ffff700 1 -- 172.21.15.196:0/347170304 wait complete. 2020-05-18T21:40:26.535 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc34ffff700 1 Processor -- start 2020-05-18T21:40:26.535 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc34ffff700 1 -- start start 2020-05-18T21:40:26.535 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc34ffff700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc350024ba0 0x7fc3500644d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:26.536 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc34ffff700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc35005f260 0x7fc3501322a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:26.536 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc34ffff700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3500a4c40 con 0x7fc3500a68d0 2020-05-18T21:40:26.536 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc34ffff700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3500a5730 con 0x7fc35005f260 2020-05-18T21:40:26.536 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc34ffff700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3500a51f0 con 0x7fc350024ba0 2020-05-18T21:40:26.537 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc347fff700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc35005f260 0x7fc3501322a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:26.537 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc347fff700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc35005f260 0x7fc3501322a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.196:38328/0 (socket says 172.21.15.196:38328) 2020-05-18T21:40:26.537 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc34f7fe700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc350024ba0 0x7fc3500644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:26.537 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc347fff700 1 -- 172.21.15.196:0/1358156754 learned_addr learned my addr 172.21.15.196:0/1358156754 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:26.538 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc3553a8700 1 --1- 172.21.15.196:0/1358156754 >> v1:172.21.15.80:6789/0 conn(0x7fc3500a68d0 0x7fc3500a3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.196:38692/0 (socket says 172.21.15.196:38692) 2020-05-18T21:40:26.540 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc347fff700 1 -- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc350024ba0 msgr2=0x7fc3500644d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:26.541 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc347fff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc350024ba0 0x7fc3500644d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:26.541 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc347fff700 1 -- 172.21.15.196:0/1358156754 >> v1:172.21.15.80:6789/0 conn(0x7fc3500a68d0 legacy=0x7fc3500a3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:26.541 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc347fff700 1 -- 172.21.15.196:0/1358156754 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3500a5490 con 0x7fc35005f260 2020-05-18T21:40:26.541 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc347fff700 1 --2- 172.21.15.196:0/1358156754 >> v2:172.21.15.80:3300/0 conn(0x7fc35005f260 0x7fc3501322a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc338000bc0 tx=0x7fc338010980).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.542 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc347fff700 1 --2- 172.21.15.196:0/1358156754 >> v2:172.21.15.80:3300/0 conn(0x7fc35005f260 0x7fc3501322a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc338000bc0 tx=0x7fc338010980).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.542 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc347fff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc35005f260 0x7fc3501322a0 secure :-1 s=READY pgs=56 cs=0 l=1 rx=0x7fc338000bc0 tx=0x7fc338010980).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:26.542 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc347fff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc35005f260 0x7fc3501322a0 secure :-1 s=THROTTLE_DONE pgs=56 cs=0 l=1 rx=0x7fc338000bc0 tx=0x7fc338010980).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.542 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc34cff9700 1 -- 172.21.15.196:0/1358156754 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc338004910 con 0x7fc35005f260 2020-05-18T21:40:26.542 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc347fff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc35005f260 0x7fc3501322a0 secure :-1 s=THROTTLE_DONE pgs=56 cs=0 l=1 rx=0x7fc338000bc0 tx=0x7fc338010980).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.543 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc34ffff700 1 -- 172.21.15.196:0/1358156754 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3500ac0f0 con 0x7fc35005f260 2020-05-18T21:40:26.543 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc34cff9700 1 -- 172.21.15.196:0/1358156754 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc338004a70 con 0x7fc35005f260 2020-05-18T21:40:26.543 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc347fff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc35005f260 0x7fc3501322a0 secure :-1 s=THROTTLE_DONE pgs=56 cs=0 l=1 rx=0x7fc338000bc0 tx=0x7fc338010980).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.543 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc34cff9700 1 -- 172.21.15.196:0/1358156754 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc338004e50 con 0x7fc35005f260 2020-05-18T21:40:26.544 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.527+0000 7fc34ffff700 1 -- 172.21.15.196:0/1358156754 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35007f0b0 con 0x7fc35005f260 2020-05-18T21:40:26.544 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.531+0000 7fc347fff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc35005f260 0x7fc3501322a0 secure :-1 s=THROTTLE_DONE pgs=56 cs=0 l=1 rx=0x7fc338000bc0 tx=0x7fc338010980).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.544 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.531+0000 7fc34ffff700 1 -- 172.21.15.196:0/1358156754 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3500569e0 con 0x7fc35005f260 2020-05-18T21:40:26.544 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.531+0000 7fc347fff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc35005f260 0x7fc3501322a0 secure :-1 s=THROTTLE_DONE pgs=56 cs=0 l=1 rx=0x7fc338000bc0 tx=0x7fc338010980).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.544 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.531+0000 7fc34cff9700 1 -- 172.21.15.196:0/1358156754 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fc338012350 con 0x7fc35005f260 2020-05-18T21:40:26.545 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.531+0000 7fc34cff9700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc33404f8f0 0x7fc334051db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:26.545 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.531+0000 7fc34cff9700 1 -- 172.21.15.196:0/1358156754 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7fc338063b90 con 0x7fc35005f260 2020-05-18T21:40:26.545 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.531+0000 7fc34f7fe700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc33404f8f0 0x7fc334051db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:26.545 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.531+0000 7fc34f7fe700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc33404f8f0 0x7fc334051db0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc340002f70 tx=0x7fc340007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.545 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.531+0000 7fc34f7fe700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc33404f8f0 0x7fc334051db0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc340002f70 tx=0x7fc340007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.546 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.531+0000 7fc34f7fe700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc33404f8f0 0x7fc334051db0 secure :-1 s=READY pgs=25 cs=0 l=1 rx=0x7fc340002f70 tx=0x7fc340007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:26.546 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.539+0000 7fc347fff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc35005f260 0x7fc3501322a0 secure :-1 s=THROTTLE_DONE pgs=56 cs=0 l=1 rx=0x7fc338000bc0 tx=0x7fc338010980).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.546 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.539+0000 7fc34cff9700 1 -- 172.21.15.196:0/1358156754 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7fc33803de80 con 0x7fc35005f260 2020-05-18T21:40:26.661 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.663+0000 7fc34ffff700 1 -- 172.21.15.196:0/1358156754 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc350067b30 con 0x7fc35005f260 2020-05-18T21:40:26.663 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.663+0000 7fc347fff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc35005f260 0x7fc3501322a0 secure :-1 s=THROTTLE_DONE pgs=56 cs=0 l=1 rx=0x7fc338000bc0 tx=0x7fc338010980).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.663 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.663+0000 7fc34cff9700 1 -- 172.21.15.196:0/1358156754 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+11856 (secure 0 0 0) 0x7fc3380414a0 con 0x7fc35005f260 2020-05-18T21:40:26.664 INFO:teuthology.misc.health.smithi196.stdout: 2020-05-18T21:40:26.665 INFO:teuthology.misc.health.smithi196.stdout:{"epoch":15,"fsid":"d048673a-3d91-4cc6-882d-e9cc70e7cd6c","created":"2020-05-18T21:40:15.911738+0000","modified":"2020-05-18T21:40:24.769175+0000","last_up_change":"2020-05-18T21:40:22.762927+0000","last_in_change":"2020-05-18T21:40:22.762927+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-05-18T21:40:19.808941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"8db63a0a-61dc-42b5-9559-fddcf436cab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":12771},{"type":"v1","addr":"172.21.15.196:6801","nonce":12771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":12771},{"type":"v1","addr":"172.21.15.196:6803","nonce":12771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":12771},{"type":"v1","addr":"172.21.15.196:6807","nonce":12771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":12771},{"type":"v1","addr":"172.21.15.196:6805","nonce":12771}]},"public_addr":"172.21.15.196:6801/12771","cluster_addr":"172.21.15.196:6803/12771","heartbeat_back_addr":"172.21.15.196:6807/12771","heartbeat_front_addr":"172.21.15.196:6805/12771","state":["exists","up"]},{"osd":1,"uuid":"6773b0c3-9f49-42a2-b75b-c10c734ff086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":12772},{"type":"v1","addr":"172.21.15.196:6817","nonce":12772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":12772},{"type":"v1","addr":"172.21.15.196:6819","nonce":12772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":12772},{"type":"v1","addr":"172.21.15.196:6823","nonce":12772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":12772},{"type":"v1","addr":"172.21.15.196:6821","nonce":12772}]},"public_addr":"172.21.15.196:6817/12772","cluster_addr":"172.21.15.196:6819/12772","heartbeat_back_addr":"172.21.15.196:6823/12772","heartbeat_front_addr":"172.21.15.196:6821/12772","state":["exists","up"]},{"osd":2,"uuid":"ac1f2e05-9bb7-4768-b712-10e4d2ea4154","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":12773},{"type":"v1","addr":"172.21.15.196:6809","nonce":12773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":12773},{"type":"v1","addr":"172.21.15.196:6811","nonce":12773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":12773},{"type":"v1","addr":"172.21.15.196:6815","nonce":12773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":12773},{"type":"v1","addr":"172.21.15.196:6813","nonce":12773}]},"public_addr":"172.21.15.196:6809/12773","cluster_addr":"172.21.15.196:6811/12773","heartbeat_back_addr":"172.21.15.196:6815/12773","heartbeat_front_addr":"172.21.15.196:6813/12773","state":["exists","up"]},{"osd":3,"uuid":"bae5ab40-235b-4e0f-8d0d-524afd099b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":12775},{"type":"v1","addr":"172.21.15.196:6825","nonce":12775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":12775},{"type":"v1","addr":"172.21.15.196:6827","nonce":12775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":12775},{"type":"v1","addr":"172.21.15.196:6831","nonce":12775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":12775},{"type":"v1","addr":"172.21.15.196:6829","nonce":12775}]},"public_addr":"172.21.15.196:6825/12775","cluster_addr":"172.21.15.196:6827/12775","heartbeat_back_addr":"172.21.15.196:6831/12775","heartbeat_front_addr":"172.21.15.196:6829/12775","state":["exists","up"]},{"osd":4,"uuid":"275fc52c-58e4-4f35-a0cb-e117fe36bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":13040},{"type":"v1","addr":"172.21.15.80:6819","nonce":13040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6821","nonce":13040},{"type":"v1","addr":"172.21.15.80:6823","nonce":13040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6829","nonce":13040},{"type":"v1","addr":"172.21.15.80:6831","nonce":13040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":13040},{"type":"v1","addr":"172.21.15.80:6827","nonce":13040}]},"public_addr":"172.21.15.80:6819/13040","cluster_addr":"172.21.15.80:6823/13040","heartbeat_back_addr":"172.21.15.80:6831/13040","heartbeat_front_addr":"172.21.15.80:6827/13040","state":["exists","up"]},{"osd":5,"uuid":"9795b89c-e2f8-4abf-b9c7-796727a57002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":13037},{"type":"v1","addr":"172.21.15.80:6818","nonce":13037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":13037},{"type":"v1","addr":"172.21.15.80:6822","nonce":13037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":13037},{"type":"v1","addr":"172.21.15.80:6830","nonce":13037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":13037},{"type":"v1","addr":"172.21.15.80:6826","nonce":13037}]},"public_addr":"172.21.15.80:6818/13037","cluster_addr":"172.21.15.80:6822/13037","heartbeat_back_addr":"172.21.15.80:6830/13037","heartbeat_front_addr":"172.21.15.80:6826/13037","state":["exists","up"]},{"osd":6,"uuid":"e42df2e8-9cc5-4c7e-a243-bd18ccfbd002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":13039},{"type":"v1","addr":"172.21.15.80:6804","nonce":13039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":13039},{"type":"v1","addr":"172.21.15.80:6808","nonce":13039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":13039},{"type":"v1","addr":"172.21.15.80:6813","nonce":13039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":13039},{"type":"v1","addr":"172.21.15.80:6810","nonce":13039}]},"public_addr":"172.21.15.80:6804/13039","cluster_addr":"172.21.15.80:6808/13039","heartbeat_back_addr":"172.21.15.80:6813/13039","heartbeat_front_addr":"172.21.15.80:6810/13039","state":["exists","up"]},{"osd":7,"uuid":"ebd6c938-beff-4668-b70e-076ed22bb8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":13038},{"type":"v1","addr":"172.21.15.80:6801","nonce":13038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":13038},{"type":"v1","addr":"172.21.15.80:6805","nonce":13038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":13038},{"type":"v1","addr":"172.21.15.80:6817","nonce":13038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":13038},{"type":"v1","addr":"172.21.15.80:6811","nonce":13038}]},"public_addr":"172.21.15.80:6801/13038","cluster_addr":"172.21.15.80:6805/13038","heartbeat_back_addr":"172.21.15.80:6817/13038","heartbeat_front_addr":"172.21.15.80:6811/13038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.134515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.439836+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.476825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.176642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.452301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.427183+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.230250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.210168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-18T21:40:26.666 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.663+0000 7fc34ffff700 1 -- 172.21.15.196:0/1358156754 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc33404f8f0 msgr2=0x7fc334051db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:26.666 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.663+0000 7fc34ffff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc33404f8f0 0x7fc334051db0 secure :-1 s=READY pgs=25 cs=0 l=1 rx=0x7fc340002f70 tx=0x7fc340007a80).stop 2020-05-18T21:40:26.666 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.663+0000 7fc34ffff700 1 -- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc35005f260 msgr2=0x7fc3501322a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:26.666 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.663+0000 7fc34ffff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc35005f260 0x7fc3501322a0 secure :-1 s=READY pgs=56 cs=0 l=1 rx=0x7fc338000bc0 tx=0x7fc338010980).stop 2020-05-18T21:40:26.667 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.667+0000 7fc34ffff700 1 -- 172.21.15.196:0/1358156754 shutdown_connections 2020-05-18T21:40:26.667 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.667+0000 7fc34ffff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc35005f260 0x7fc3501322a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:26.667 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.667+0000 7fc34ffff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc33404f8f0 0x7fc334051db0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:26.668 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.667+0000 7fc34ffff700 1 --2- 172.21.15.196:0/1358156754 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc350024ba0 0x7fc3500644d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:26.668 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.667+0000 7fc34ffff700 1 -- 172.21.15.196:0/1358156754 >> 172.21.15.196:0/1358156754 conn(0x7fc3500a8460 msgr2=0x7fc35005bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:26.668 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.667+0000 7fc34ffff700 1 -- 172.21.15.196:0/1358156754 shutdown_connections 2020-05-18T21:40:26.668 INFO:teuthology.misc.health.smithi196.stderr:2020-05-18T21:40:26.667+0000 7fc34ffff700 1 -- 172.21.15.196:0/1358156754 wait complete. 2020-05-18T21:40:26.675 DEBUG:teuthology.misc:8 of 8 OSDs are up 2020-05-18T21:40:26.675 INFO:tasks.ceph:Creating RBD pool 2020-05-18T21:40:26.676 INFO:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd pool create rbd 8 2020-05-18T21:40:26.790 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f1192142700 1 Processor -- start 2020-05-18T21:40:26.790 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f1192142700 1 -- start start 2020-05-18T21:40:26.790 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f1192142700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f118c14ddc0 0x7f118c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f1192142700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c0a6590 con 0x7f118c0a7ba0 2020-05-18T21:40:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f1192142700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c0ac3f0 con 0x7f118c0abb40 2020-05-18T21:40:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f1192142700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118c07fec0 con 0x7f118c14ddc0 2020-05-18T21:40:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f1191941700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f118c0abb40 0x7f118c0a3470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42870/0 (socket says 172.21.15.196:42870) 2020-05-18T21:40:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f1191941700 1 -- 172.21.15.196:0/1420075515 learned_addr learned my addr 172.21.15.196:0/1420075515 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:26.792 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f11933a4700 1 --2- 172.21.15.196:0/1420075515 >> v2:172.21.15.80:3300/0 conn(0x7f118c14ddc0 0x7f118c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:26.792 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773745732 0 0) 0x7f118c0ac3f0 con 0x7f118c0abb40 2020-05-18T21:40:26.792 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1178003270 con 0x7f118c0abb40 2020-05-18T21:40:26.792 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272686670 0 0) 0x7f118c0a6590 con 0x7f118c0a7ba0 2020-05-18T21:40:26.792 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f118c0ac3f0 con 0x7f118c0a7ba0 2020-05-18T21:40:26.793 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2265427652 0 0) 0x7f1178003270 con 0x7f118c0abb40 2020-05-18T21:40:26.793 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 >> v2:172.21.15.80:3300/0 conn(0x7f118c14ddc0 msgr2=0x7f118c1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:26.793 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f119093f700 1 --2- 172.21.15.196:0/1420075515 >> v2:172.21.15.80:3300/0 conn(0x7f118c14ddc0 0x7f118c1501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:26.793 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 >> v1:172.21.15.80:6789/0 conn(0x7f118c0a7ba0 legacy=0x7f118c14c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:26.793 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c0a67c0 con 0x7f118c0abb40 2020-05-18T21:40:26.794 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7f1188001710 con 0x7f118c0abb40 2020-05-18T21:40:26.794 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 >> v1:172.21.15.196:6789/0 conn(0x7f118c0abb40 legacy=0x7f118c0a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:26.794 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f119093f700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11780038f0 0x7f1178003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:26.794 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.779+0000 7f119093f700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1178004510 0x7f1178004990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:26.794 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f119093f700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f117804f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:26.795 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c07fec0 con 0x7f1178004510 2020-05-18T21:40:26.795 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11780053e0 con 0x7f1178004df0 2020-05-18T21:40:26.795 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117804fc50 con 0x7f11780038f0 2020-05-18T21:40:26.795 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f1191941700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11780038f0 0x7f1178003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:26.795 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f11933a4700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f117804f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:26.796 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f1191140700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1178004510 0x7f1178004990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:26.796 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f11933a4700 1 -- 172.21.15.196:0/1420075515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11780038f0 msgr2=0x7f1178003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:26.796 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f11933a4700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11780038f0 0x7f1178003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:26.796 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f11933a4700 1 -- 172.21.15.196:0/1420075515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1178004510 msgr2=0x7f1178004990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:26.796 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f11933a4700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1178004510 0x7f1178004990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:26.797 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f11933a4700 1 -- 172.21.15.196:0/1420075515 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1178003270 con 0x7f1178004df0 2020-05-18T21:40:26.797 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f1191941700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11780038f0 0x7f1178003cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:26.797 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f1191140700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1178004510 0x7f1178004990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:26.797 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f11933a4700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f117804f770 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f11840032f0 tx=0x7f11840103e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.797 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f11933a4700 1 -- 172.21.15.196:0/1420075515 reap_dead start 2020-05-18T21:40:26.798 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f11933a4700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f117804f770 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f11840032f0 tx=0x7f11840103e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.798 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f11933a4700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f117804f770 secure :-1 s=READY pgs=63 cs=0 l=1 rx=0x7f11840032f0 tx=0x7f11840103e0).ready entity=mon.1 client_cookie=133d54b7f214fa93 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:26.798 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f11933a4700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f117804f770 secure :-1 s=THROTTLE_DONE pgs=63 cs=0 l=1 rx=0x7f11840032f0 tx=0x7f11840103e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.798 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f11933a4700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f117804f770 secure :-1 s=THROTTLE_DONE pgs=63 cs=0 l=1 rx=0x7f11840032f0 tx=0x7f11840103e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.799 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1184010e50 con 0x7f1178004df0 2020-05-18T21:40:26.799 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f11933a4700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f117804f770 secure :-1 s=THROTTLE_DONE pgs=63 cs=0 l=1 rx=0x7f11840032f0 tx=0x7f11840103e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.799 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1184005c70 con 0x7f1178004df0 2020-05-18T21:40:26.799 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f119093f700 1 -- 172.21.15.196:0/1420075515 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11840110d0 con 0x7f1178004df0 2020-05-18T21:40:26.799 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f1192142700 1 -- 172.21.15.196:0/1420075515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 msgr2=0x7f117804f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:26.800 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f1192142700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f117804f770 secure :-1 s=READY pgs=63 cs=0 l=1 rx=0x7f11840032f0 tx=0x7f11840103e0).stop 2020-05-18T21:40:26.800 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f1192142700 1 -- 172.21.15.196:0/1420075515 shutdown_connections 2020-05-18T21:40:26.800 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f1192142700 1 --2- 172.21.15.196:0/1420075515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f117804f770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:26.800 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f1192142700 1 -- 172.21.15.196:0/1420075515 >> 172.21.15.196:0/1420075515 conn(0x7f118c069ac0 msgr2=0x7f118c069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:26.800 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f1192142700 1 -- 172.21.15.196:0/1420075515 shutdown_connections 2020-05-18T21:40:26.800 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.783+0000 7f1192142700 1 -- 172.21.15.196:0/1420075515 wait complete. 2020-05-18T21:40:26.801 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1192142700 1 Processor -- start 2020-05-18T21:40:26.801 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1192142700 1 -- start start 2020-05-18T21:40:26.801 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1192142700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1178004df0 0x7f118c0a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:26.801 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1192142700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f118c07e450 0x7f118c07e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:26.801 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1192142700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c1274a0 con 0x7f118c128b80 2020-05-18T21:40:26.802 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1192142700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118c0a9dc0 con 0x7f1178004df0 2020-05-18T21:40:26.802 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1192142700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0aa810 con 0x7f118c07e450 2020-05-18T21:40:26.802 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1178004df0 0x7f118c0a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:26.802 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1178004df0 0x7f118c0a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34122/0 (socket says 172.21.15.196:34122) 2020-05-18T21:40:26.802 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 -- 172.21.15.196:0/3835484780 learned_addr learned my addr 172.21.15.196:0/3835484780 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:26.803 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191140700 1 --1- 172.21.15.196:0/3835484780 >> v1:172.21.15.196:6789/0 conn(0x7f118c128b80 0x7f118c137660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42884/0 (socket says 172.21.15.196:42884) 2020-05-18T21:40:26.803 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f11933a4700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f118c07e450 0x7f118c07e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:26.803 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 -- 172.21.15.196:0/3835484780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f118c07e450 msgr2=0x7f118c07e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:26.803 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f118c07e450 0x7f118c07e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:26.803 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 -- 172.21.15.196:0/3835484780 >> v1:172.21.15.196:6789/0 conn(0x7f118c128b80 legacy=0x7f118c137660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:26.804 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 -- 172.21.15.196:0/3835484780 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c129100 con 0x7f1178004df0 2020-05-18T21:40:26.804 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f11827fc700 1 -- 172.21.15.196:0/3835484780 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634828879 0 0) 0x7f118c1274a0 con 0x7f118c128b80 2020-05-18T21:40:26.804 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 --2- 172.21.15.196:0/3835484780 >> v2:172.21.15.196:3300/0 conn(0x7f1178004df0 0x7f118c0a1fa0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f11880018f0 tx=0x7f118c0a3be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.804 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 --2- 172.21.15.196:0/3835484780 >> v2:172.21.15.196:3300/0 conn(0x7f1178004df0 0x7f118c0a1fa0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f11880018f0 tx=0x7f118c0a3be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.804 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f118c0a1fa0 secure :-1 s=READY pgs=65 cs=0 l=1 rx=0x7f11880018f0 tx=0x7f118c0a3be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:26.805 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f118c0a1fa0 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f11880018f0 tx=0x7f118c0a3be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.805 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f11827fc700 1 -- 172.21.15.196:0/3835484780 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1188011e50 con 0x7f1178004df0 2020-05-18T21:40:26.805 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f118c0a1fa0 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f11880018f0 tx=0x7f118c0a3be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.805 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f118c0a1fa0 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f11880018f0 tx=0x7f118c0a3be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.806 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f11827fc700 1 -- 172.21.15.196:0/3835484780 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118800c7c0 con 0x7f1178004df0 2020-05-18T21:40:26.806 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1192142700 1 -- 172.21.15.196:0/3835484780 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c0aaa00 con 0x7f1178004df0 2020-05-18T21:40:26.806 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f11827fc700 1 -- 172.21.15.196:0/3835484780 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f118800cb20 con 0x7f1178004df0 2020-05-18T21:40:26.806 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1192142700 1 -- 172.21.15.196:0/3835484780 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c0a2880 con 0x7f1178004df0 2020-05-18T21:40:26.807 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.787+0000 7f1191941700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f118c0a1fa0 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f11880018f0 tx=0x7f118c0a3be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.807 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.791+0000 7f1192142700 1 -- 172.21.15.196:0/3835484780 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c0569e0 con 0x7f1178004df0 2020-05-18T21:40:26.807 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.791+0000 7f11827fc700 1 -- 172.21.15.196:0/3835484780 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f118800cc80 con 0x7f1178004df0 2020-05-18T21:40:26.807 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.791+0000 7f1191941700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f118c0a1fa0 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f11880018f0 tx=0x7f118c0a3be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.808 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.791+0000 7f11827fc700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f117404f870 0x7f1174051d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:26.809 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.791+0000 7f1191140700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f117404f870 0x7f1174051d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:26.809 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.791+0000 7f11827fc700 1 -- 172.21.15.196:0/3835484780 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f1188063bc0 con 0x7f1178004df0 2020-05-18T21:40:26.809 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.791+0000 7f1191140700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f117404f870 0x7f1174051d30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f117c001230 tx=0x7f117c00cb50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.809 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.791+0000 7f1191140700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f117404f870 0x7f1174051d30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f117c001230 tx=0x7f117c00cb50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.810 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.791+0000 7f1191140700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f117404f870 0x7f1174051d30 secure :-1 s=READY pgs=26 cs=0 l=1 rx=0x7f117c001230 tx=0x7f117c00cb50).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:26.810 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.795+0000 7f1191941700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f118c0a1fa0 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f11880018f0 tx=0x7f118c0a3be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:26.810 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.795+0000 7f11827fc700 1 -- 172.21.15.196:0/3835484780 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f118803def0 con 0x7f1178004df0 2020-05-18T21:40:26.912 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:26.911+0000 7f1192142700 1 -- 172.21.15.196:0/3835484780 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f118c05c940 con 0x7f1178004df0 2020-05-18T21:40:27.787 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.787+0000 7f1191941700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f118c0a1fa0 secure :-1 s=READY pgs=65 cs=0 l=1 rx=0x7f11880018f0 tx=0x7f118c0a3be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:27.858 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.859+0000 7f1191941700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f118c0a1fa0 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f11880018f0 tx=0x7f118c0a3be0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:27.859 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.859+0000 7f11827fc700 1 -- 172.21.15.196:0/3835484780 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7f1188041510 con 0x7f1178004df0 2020-05-18T21:40:27.863 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.863+0000 7f116e7fc700 1 -- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f117404f870 msgr2=0x7f1174051d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:27.863 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.863+0000 7f116e7fc700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f117404f870 0x7f1174051d30 secure :-1 s=READY pgs=26 cs=0 l=1 rx=0x7f117c001230 tx=0x7f117c00cb50).stop 2020-05-18T21:40:27.863 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.863+0000 7f116e7fc700 1 -- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 msgr2=0x7f118c0a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:27.864 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.863+0000 7f116e7fc700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f118c0a1fa0 secure :-1 s=READY pgs=65 cs=0 l=1 rx=0x7f11880018f0 tx=0x7f118c0a3be0).stop 2020-05-18T21:40:27.864 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.863+0000 7f116e7fc700 1 -- 172.21.15.196:0/3835484780 shutdown_connections 2020-05-18T21:40:27.864 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.863+0000 7f116e7fc700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f117404f870 0x7f1174051d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:27.864 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.863+0000 7f116e7fc700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f118c07e450 0x7f118c07e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:27.864 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.863+0000 7f116e7fc700 1 --2- 172.21.15.196:0/3835484780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1178004df0 0x7f118c0a1fa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:27.865 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.863+0000 7f116e7fc700 1 -- 172.21.15.196:0/3835484780 >> 172.21.15.196:0/3835484780 conn(0x7f118c069ac0 msgr2=0x7f118c05eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:27.865 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.863+0000 7f116e7fc700 1 -- 172.21.15.196:0/3835484780 shutdown_connections 2020-05-18T21:40:27.865 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.863+0000 7f116e7fc700 1 -- 172.21.15.196:0/3835484780 wait complete. 2020-05-18T21:40:27.865 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'rbd' created 2020-05-18T21:40:27.874 INFO:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-05-18T21:40:27.981 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.975+0000 7f71b09cf700 1 Processor -- start 2020-05-18T21:40:27.981 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.975+0000 7f71b09cf700 1 -- start start 2020-05-18T21:40:27.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.975+0000 7f71b09cf700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71ac07fda0 0x7f71ac144e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:27.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.975+0000 7f71b09cf700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f71ac145260 0x7f71ac14f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:27.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.975+0000 7f71b09cf700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ac0255f0 con 0x7f71ac024ab0 2020-05-18T21:40:27.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.975+0000 7f71b09cf700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac0a5d40 con 0x7f71ac145260 2020-05-18T21:40:27.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.975+0000 7f71b09cf700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac024e90 con 0x7f71ac07fda0 2020-05-18T21:40:27.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71b1c31700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f71ac145260 0x7f71ac14f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:27.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71ab7fe700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71ac07fda0 0x7f71ac144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:27.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71abfff700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f71ac024ab0 0x7f71ac07f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.196:38718/0 (socket says 172.21.15.196:38718) 2020-05-18T21:40:27.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71abfff700 1 -- 172.21.15.196:0/2587335370 learned_addr learned my addr 172.21.15.196:0/2587335370 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:27.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71aaffd700 1 -- 172.21.15.196:0/2587335370 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115771544 0 0) 0x7f71ac0255f0 con 0x7f71ac024ab0 2020-05-18T21:40:27.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71aaffd700 1 -- 172.21.15.196:0/2587335370 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7198003270 con 0x7f71ac024ab0 2020-05-18T21:40:27.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71ab7fe700 1 -- 172.21.15.196:0/2587335370 >> v2:172.21.15.80:3300/0 conn(0x7f71ac145260 msgr2=0x7f71ac14f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:27.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/2587335370 >> v2:172.21.15.80:3300/0 conn(0x7f71ac145260 0x7f71ac14f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:27.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71ab7fe700 1 -- 172.21.15.196:0/2587335370 >> v1:172.21.15.80:6789/0 conn(0x7f71ac024ab0 legacy=0x7f71ac07f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:27.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71ab7fe700 1 -- 172.21.15.196:0/2587335370 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac0250b0 con 0x7f71ac07fda0 2020-05-18T21:40:27.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71b1c31700 1 --2- 172.21.15.196:0/2587335370 >> v2:172.21.15.80:3300/0 conn(0x7f71ac145260 0x7f71ac14f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:27.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/2587335370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71ac07fda0 0x7f71ac144e00 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:27.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/2587335370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71ac07fda0 0x7f71ac144e00 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:27.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/2587335370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71ac07fda0 0x7f71ac144e00 secure :-1 s=READY pgs=20 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).ready entity=mon.2 client_cookie=1e10a03232c88697 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:27.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/2587335370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71ac07fda0 0x7f71ac144e00 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:27.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71aaffd700 1 -- 172.21.15.196:0/2587335370 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7194008e80 con 0x7f71ac07fda0 2020-05-18T21:40:27.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/2587335370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71ac07fda0 0x7f71ac144e00 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:27.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/2587335370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71ac07fda0 0x7f71ac144e00 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:27.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71aaffd700 1 -- 172.21.15.196:0/2587335370 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7194011540 con 0x7f71ac07fda0 2020-05-18T21:40:27.991 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71aaffd700 1 -- 172.21.15.196:0/2587335370 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7194011960 con 0x7f71ac07fda0 2020-05-18T21:40:27.991 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71b09cf700 1 -- 172.21.15.196:0/2587335370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71ac07fda0 msgr2=0x7f71ac144e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:27.991 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71b09cf700 1 --2- 172.21.15.196:0/2587335370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71ac07fda0 0x7f71ac144e00 secure :-1 s=READY pgs=20 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).stop 2020-05-18T21:40:27.991 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71b09cf700 1 -- 172.21.15.196:0/2587335370 shutdown_connections 2020-05-18T21:40:27.991 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71b09cf700 1 --2- 172.21.15.196:0/2587335370 >> v2:172.21.15.80:3300/0 conn(0x7f71ac145260 0x7f71ac14f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:27.992 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71b09cf700 1 --2- 172.21.15.196:0/2587335370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71ac07fda0 0x7f71ac144e00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:27.992 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71b09cf700 1 -- 172.21.15.196:0/2587335370 >> 172.21.15.196:0/2587335370 conn(0x7f71ac069ac0 msgr2=0x7f71ac069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:27.992 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71b09cf700 1 -- 172.21.15.196:0/2587335370 shutdown_connections 2020-05-18T21:40:27.992 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.979+0000 7f71b09cf700 1 -- 172.21.15.196:0/2587335370 wait complete. 2020-05-18T21:40:27.995 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71b09cf700 1 Processor -- start 2020-05-18T21:40:27.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71b09cf700 1 -- start start 2020-05-18T21:40:27.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71b09cf700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f71ac145260 0x7f71ac1223a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:27.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71b09cf700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f71ac0abef0 0x7f71ac131510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:27.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71b09cf700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ac0aa5b0 con 0x7f71ac024ab0 2020-05-18T21:40:27.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71b09cf700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac0aae90 con 0x7f71ac145260 2020-05-18T21:40:27.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71b09cf700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac0aa850 con 0x7f71ac0abef0 2020-05-18T21:40:27.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71abfff700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f71ac024ab0 0x7f71ac09e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42896/0 (socket says 172.21.15.196:42896) 2020-05-18T21:40:27.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71abfff700 1 -- 172.21.15.196:0/1297771689 learned_addr learned my addr 172.21.15.196:0/1297771689 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:27.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71b1c31700 1 --2- 172.21.15.196:0/1297771689 >> v2:172.21.15.196:3300/0 conn(0x7f71ac0abef0 0x7f71ac131510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:27.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/1297771689 >> v2:172.21.15.80:3300/0 conn(0x7f71ac145260 0x7f71ac1223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:27.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71a8ff9700 1 -- 172.21.15.196:0/1297771689 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449958915 0 0) 0x7f71ac0aa5b0 con 0x7f71ac024ab0 2020-05-18T21:40:27.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71a8ff9700 1 -- 172.21.15.196:0/1297771689 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7190003270 con 0x7f71ac024ab0 2020-05-18T21:40:27.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71ab7fe700 1 -- 172.21.15.196:0/1297771689 >> v2:172.21.15.196:3300/0 conn(0x7f71ac0abef0 msgr2=0x7f71ac131510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:27.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/1297771689 >> v2:172.21.15.196:3300/0 conn(0x7f71ac0abef0 0x7f71ac131510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:27.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71ab7fe700 1 -- 172.21.15.196:0/1297771689 >> v1:172.21.15.196:6789/0 conn(0x7f71ac024ab0 legacy=0x7f71ac09e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:27.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71ab7fe700 1 -- 172.21.15.196:0/1297771689 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac0aaaf0 con 0x7f71ac145260 2020-05-18T21:40:27.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71b1c31700 1 --2- 172.21.15.196:0/1297771689 >> v2:172.21.15.196:3300/0 conn(0x7f71ac0abef0 0x7f71ac131510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:27.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/1297771689 >> v2:172.21.15.80:3300/0 conn(0x7f71ac145260 0x7f71ac1223a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:27.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/1297771689 >> v2:172.21.15.80:3300/0 conn(0x7f71ac145260 0x7f71ac1223a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.983+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71ac145260 0x7f71ac1223a0 secure :-1 s=READY pgs=59 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:28.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71ac145260 0x7f71ac1223a0 secure :-1 s=THROTTLE_DONE pgs=59 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71a8ff9700 1 -- 172.21.15.196:0/1297771689 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7194011ea0 con 0x7f71ac145260 2020-05-18T21:40:28.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71ac145260 0x7f71ac1223a0 secure :-1 s=THROTTLE_DONE pgs=59 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71a8ff9700 1 -- 172.21.15.196:0/1297771689 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719400f860 con 0x7f71ac145260 2020-05-18T21:40:28.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71ac145260 0x7f71ac1223a0 secure :-1 s=THROTTLE_DONE pgs=59 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71a8ff9700 1 -- 172.21.15.196:0/1297771689 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7194007550 con 0x7f71ac145260 2020-05-18T21:40:28.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71b09cf700 1 -- 172.21.15.196:0/1297771689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac0a98e0 con 0x7f71ac145260 2020-05-18T21:40:28.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71b09cf700 1 -- 172.21.15.196:0/1297771689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac0ab650 con 0x7f71ac145260 2020-05-18T21:40:28.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71ac145260 0x7f71ac1223a0 secure :-1 s=THROTTLE_DONE pgs=59 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71b09cf700 1 -- 172.21.15.196:0/1297771689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac131cb0 con 0x7f71ac145260 2020-05-18T21:40:28.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71a8ff9700 1 -- 172.21.15.196:0/1297771689 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f7194005700 con 0x7f71ac145260 2020-05-18T21:40:28.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71ac145260 0x7f71ac1223a0 secure :-1 s=THROTTLE_DONE pgs=59 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71a8ff9700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f719004fd40 0x7f7190052200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:28.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71a8ff9700 1 -- 172.21.15.196:0/1297771689 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5242+0+0 (secure 0 0 0) 0x7f7194064430 con 0x7f71ac145260 2020-05-18T21:40:28.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.987+0000 7f71abfff700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f719004fd40 0x7f7190052200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:28.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.991+0000 7f71abfff700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f719004fd40 0x7f7190052200 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f719c0014d0 tx=0x7f719c001a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.991+0000 7f71abfff700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f719004fd40 0x7f7190052200 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f719c0014d0 tx=0x7f719c001a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.991+0000 7f71abfff700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f719004fd40 0x7f7190052200 secure :-1 s=READY pgs=27 cs=0 l=1 rx=0x7f719c0014d0 tx=0x7f719c001a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:28.004 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.999+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71ac145260 0x7f71ac1223a0 secure :-1 s=THROTTLE_DONE pgs=59 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.004 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:27.999+0000 7f71a8ff9700 1 -- 172.21.15.196:0/1297771689 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f719403e820 con 0x7f71ac145260 2020-05-18T21:40:28.116 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.115+0000 7f71b09cf700 1 -- 172.21.15.196:0/1297771689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f71ac0569e0 con 0x7f71ac145260 2020-05-18T21:40:28.861 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.859+0000 7f71ab7fe700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71ac145260 0x7f71ac1223a0 secure :-1 s=THROTTLE_DONE pgs=59 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.862 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.859+0000 7f71a8ff9700 1 -- 172.21.15.196:0/1297771689 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7f7194041e40 con 0x7f71ac145260 2020-05-18T21:40:28.868 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.867+0000 7f71b09cf700 1 -- 172.21.15.196:0/1297771689 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f719004fd40 msgr2=0x7f7190052200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:28.869 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.867+0000 7f71b09cf700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f719004fd40 0x7f7190052200 secure :-1 s=READY pgs=27 cs=0 l=1 rx=0x7f719c0014d0 tx=0x7f719c001a80).stop 2020-05-18T21:40:28.869 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.867+0000 7f71b09cf700 1 -- 172.21.15.196:0/1297771689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71ac145260 msgr2=0x7f71ac1223a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:28.869 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.867+0000 7f71b09cf700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71ac145260 0x7f71ac1223a0 secure :-1 s=READY pgs=59 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).stop 2020-05-18T21:40:28.870 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.867+0000 7f71b09cf700 1 -- 172.21.15.196:0/1297771689 shutdown_connections 2020-05-18T21:40:28.870 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.867+0000 7f71b09cf700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f719004fd40 0x7f7190052200 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:28.870 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.867+0000 7f71b09cf700 1 --2- 172.21.15.196:0/1297771689 >> v2:172.21.15.196:3300/0 conn(0x7f71ac0abef0 0x7f71ac131510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:28.871 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.867+0000 7f71b09cf700 1 --2- 172.21.15.196:0/1297771689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71ac145260 0x7f71ac1223a0 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rx=0x7f7194002f70 tx=0x7f7194007a80).stop 2020-05-18T21:40:28.871 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.867+0000 7f71b09cf700 1 -- 172.21.15.196:0/1297771689 >> 172.21.15.196:0/1297771689 conn(0x7f71ac069ac0 msgr2=0x7f71ac0a8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:28.871 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.871+0000 7f71b09cf700 1 -- 172.21.15.196:0/1297771689 shutdown_connections 2020-05-18T21:40:28.871 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.871+0000 7f71b09cf700 1 -- 172.21.15.196:0/1297771689 wait complete. 2020-05-18T21:40:28.872 INFO:teuthology.orchestra.run.smithi196.stderr:enabled application 'rbd' on pool 'rbd' 2020-05-18T21:40:28.883 INFO:tasks.ceph:Setting up CephFS filesystem... 2020-05-18T21:40:28.884 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-05-18T21:40:28.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcdc557700 1 Processor -- start 2020-05-18T21:40:28.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcdc557700 1 -- start start 2020-05-18T21:40:28.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcdc557700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbcd409ec10 0x7fbcd4066340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:28.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcdc557700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbcd40a9ba0 0x7fbcd40aa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:28.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcdc557700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbcd4139680 0x7fbcd4139b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:28.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcdc557700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd40a2660 con 0x7fbcd4139680 2020-05-18T21:40:28.985 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcdc557700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd405b580 con 0x7fbcd40a9ba0 2020-05-18T21:40:28.986 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcdc557700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd405b040 con 0x7fbcd409ec10 2020-05-18T21:40:28.986 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbcd40a9ba0 0x7fbcd40aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:28.986 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcdaaf4700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbcd4139680 0x7fbcd4139b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:28.986 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbcd40a9ba0 0x7fbcd40aa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34148/0 (socket says 172.21.15.196:34148) 2020-05-18T21:40:28.986 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcdaaf4700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbcd4139680 0x7fbcd4139b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.196:38372/0 (socket says 172.21.15.196:38372) 2020-05-18T21:40:28.987 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcda2f3700 1 --2- 172.21.15.196:0/1444777649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbcd409ec10 0x7fbcd4066340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:28.987 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 -- 172.21.15.196:0/1444777649 learned_addr learned my addr 172.21.15.196:0/1444777649 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:28.987 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 -- 172.21.15.196:0/1444777649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbcd409ec10 msgr2=0x7fbcd4066340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:28.987 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 --2- 172.21.15.196:0/1444777649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbcd409ec10 0x7fbcd4066340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:28.988 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 -- 172.21.15.196:0/1444777649 >> v2:172.21.15.80:3300/0 conn(0x7fbcd4139680 msgr2=0x7fbcd4139b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:28.988 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 --2- 172.21.15.196:0/1444777649 >> v2:172.21.15.80:3300/0 conn(0x7fbcd4139680 0x7fbcd4139b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:28.988 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 -- 172.21.15.196:0/1444777649 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd405b2e0 con 0x7fbcd40a9ba0 2020-05-18T21:40:28.988 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 --2- 172.21.15.196:0/1444777649 >> v2:172.21.15.196:3300/0 conn(0x7fbcd40a9ba0 0x7fbcd40aa000 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fbcc4002f70 tx=0x7fbcc4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.989 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 --2- 172.21.15.196:0/1444777649 >> v2:172.21.15.196:3300/0 conn(0x7fbcd40a9ba0 0x7fbcd40aa000 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fbcc4002f70 tx=0x7fbcc4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.989 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 --2- 172.21.15.196:0/1444777649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd40a9ba0 0x7fbcd40aa000 secure :-1 s=READY pgs=67 cs=0 l=1 rx=0x7fbcc4002f70 tx=0x7fbcc4007a80).ready entity=mon.1 client_cookie=ee0505765f42fe0a server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:28.989 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 --2- 172.21.15.196:0/1444777649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd40a9ba0 0x7fbcd40aa000 secure :-1 s=THROTTLE_DONE pgs=67 cs=0 l=1 rx=0x7fbcc4002f70 tx=0x7fbcc4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.989 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 --2- 172.21.15.196:0/1444777649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd40a9ba0 0x7fbcd40aa000 secure :-1 s=THROTTLE_DONE pgs=67 cs=0 l=1 rx=0x7fbcc4002f70 tx=0x7fbcc4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.989 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd92f1700 1 -- 172.21.15.196:0/1444777649 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcc4011600 con 0x7fbcd40a9ba0 2020-05-18T21:40:28.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd9af2700 1 --2- 172.21.15.196:0/1444777649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd40a9ba0 0x7fbcd40aa000 secure :-1 s=THROTTLE_DONE pgs=67 cs=0 l=1 rx=0x7fbcc4002f70 tx=0x7fbcc4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:28.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd92f1700 1 -- 172.21.15.196:0/1444777649 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc4011760 con 0x7fbcd40a9ba0 2020-05-18T21:40:28.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcd92f1700 1 -- 172.21.15.196:0/1444777649 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcc4011b80 con 0x7fbcd40a9ba0 2020-05-18T21:40:28.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcdc557700 1 -- 172.21.15.196:0/1444777649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd40a9ba0 msgr2=0x7fbcd40aa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:28.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.983+0000 7fbcdc557700 1 --2- 172.21.15.196:0/1444777649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd40a9ba0 0x7fbcd40aa000 secure :-1 s=READY pgs=67 cs=0 l=1 rx=0x7fbcc4002f70 tx=0x7fbcc4007a80).stop 2020-05-18T21:40:28.995 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdc557700 1 -- 172.21.15.196:0/1444777649 shutdown_connections 2020-05-18T21:40:28.995 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdc557700 1 --2- 172.21.15.196:0/1444777649 >> v2:172.21.15.80:3300/0 conn(0x7fbcd4139680 0x7fbcd4139b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:28.995 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdc557700 1 --2- 172.21.15.196:0/1444777649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd40a9ba0 0x7fbcd40aa000 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:28.995 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdc557700 1 --2- 172.21.15.196:0/1444777649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbcd409ec10 0x7fbcd4066340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:28.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdc557700 1 -- 172.21.15.196:0/1444777649 >> 172.21.15.196:0/1444777649 conn(0x7fbcd4069ac0 msgr2=0x7fbcd4065c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:28.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdc557700 1 -- 172.21.15.196:0/1444777649 shutdown_connections 2020-05-18T21:40:28.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdc557700 1 -- 172.21.15.196:0/1444777649 wait complete. 2020-05-18T21:40:28.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdc557700 1 Processor -- start 2020-05-18T21:40:28.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdc557700 1 -- start start 2020-05-18T21:40:28.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdc557700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbcd40a9ba0 0x7fbcd4149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:28.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdc557700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd414a450 con 0x7fbcd4139680 2020-05-18T21:40:28.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdc557700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd4149a70 con 0x7fbcd409ec10 2020-05-18T21:40:28.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdc557700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd4149d10 con 0x7fbcd40a9ba0 2020-05-18T21:40:28.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcda2f3700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fbcd409ec10 0x7fbcd413a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42910/0 (socket says 172.21.15.196:42910) 2020-05-18T21:40:28.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcda2f3700 1 -- 172.21.15.196:0/3663632151 learned_addr learned my addr 172.21.15.196:0/3663632151 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:28.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcd9af2700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbcd40a9ba0 0x7fbcd4149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:28.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557328807 0 0) 0x7fbcd4149a70 con 0x7fbcd409ec10 2020-05-18T21:40:28.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcb4002fe0 con 0x7fbcd409ec10 2020-05-18T21:40:28.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627389158 0 0) 0x7fbcd414a450 con 0x7fbcd4139680 2020-05-18T21:40:28.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcd4149a70 con 0x7fbcd4139680 2020-05-18T21:40:28.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3269585789 0 0) 0x7fbcb4002fe0 con 0x7fbcd409ec10 2020-05-18T21:40:28.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 --> v1:172.21.15.196:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbcd414a450 con 0x7fbcd409ec10 2020-05-18T21:40:28.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7fbcd00014d0 con 0x7fbcd409ec10 2020-05-18T21:40:28.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbcd40a9ba0 msgr2=0x7fbcd4149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:28.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbcd40a9ba0 0x7fbcd4149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 >> v1:172.21.15.196:6789/0 conn(0x7fbcd409ec10 legacy=0x7fbcd413a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 >> v1:172.21.15.80:6789/0 conn(0x7fbcd4139680 legacy=0x7fbcd413b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbcb4004bf0 0x7fbcb4004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcb4003b30 0x7fbcb4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd4149d10 con 0x7fbcb4003b30 2020-05-18T21:40:29.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb40049b0 con 0x7fbcb4004410 2020-05-18T21:40:29.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb400bbc0 con 0x7fbcb4004bf0 2020-05-18T21:40:29.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcdaaf4700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcda2f3700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbcb4004bf0 0x7fbcb4004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.987+0000 7fbcd9af2700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcb4003b30 0x7fbcb4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdaaf4700 1 -- 172.21.15.196:0/3663632151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbcb4004bf0 msgr2=0x7fbcb4004fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdaaf4700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbcb4004bf0 0x7fbcb4004fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdaaf4700 1 -- 172.21.15.196:0/3663632151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcb4003b30 msgr2=0x7fbcb4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdaaf4700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcb4003b30 0x7fbcb4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdaaf4700 1 -- 172.21.15.196:0/3663632151 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd4149fb0 con 0x7fbcb4004410 2020-05-18T21:40:29.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdc557700 1 -- 172.21.15.196:0/3663632151 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd413ac10 con 0x7fbcb4004410 2020-05-18T21:40:29.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdaaf4700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fbccc007ce0 tx=0x7fbccc00cc90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdc557700 1 -- 172.21.15.196:0/3663632151 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd413b770 con 0x7fbcb4004410 2020-05-18T21:40:29.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdaaf4700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fbccc007ce0 tx=0x7fbccc00cc90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.004 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdaaf4700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 secure :-1 s=READY pgs=69 cs=0 l=1 rx=0x7fbccc007ce0 tx=0x7fbccc00cc90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:29.004 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdaaf4700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7fbccc007ce0 tx=0x7fbccc00cc90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.004 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbccc011e20 con 0x7fbcb4004410 2020-05-18T21:40:29.004 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdaaf4700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7fbccc007ce0 tx=0x7fbccc00cc90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.004 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc00b250 con 0x7fbcb4004410 2020-05-18T21:40:29.005 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdaaf4700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7fbccc007ce0 tx=0x7fbccc00cc90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.005 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbccc00b5f0 con 0x7fbcb4004410 2020-05-18T21:40:29.005 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdc557700 1 -- 172.21.15.196:0/3663632151 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb0003380 con 0x7fbcb4004410 2020-05-18T21:40:29.005 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdaaf4700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7fbccc007ce0 tx=0x7fbccc00cc90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.006 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fbccc00cd70 con 0x7fbcb4004410 2020-05-18T21:40:29.006 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcdaaf4700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7fbccc007ce0 tx=0x7fbccc00cc90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.006 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbccb7fe700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbcb4058300 0x7fbcb405a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.006 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fbccc063c90 con 0x7fbcb4004410 2020-05-18T21:40:29.006 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcda2f3700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbcb4058300 0x7fbcb405a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.007 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcda2f3700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbcb4058300 0x7fbcb405a7c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fbcd0007ac0 tx=0x7fbcd000d570).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.007 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcda2f3700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbcb4058300 0x7fbcb405a7c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fbcd0007ac0 tx=0x7fbcd000d570).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.007 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.991+0000 7fbcda2f3700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbcb4058300 0x7fbcb405a7c0 secure :-1 s=READY pgs=28 cs=0 l=1 rx=0x7fbcd0007ac0 tx=0x7fbcd000d570).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:29.007 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.999+0000 7fbcdaaf4700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7fbccc007ce0 tx=0x7fbccc00cc90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.008 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:28.999+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7fbccc03e080 con 0x7fbcb4004410 2020-05-18T21:40:29.115 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.115+0000 7fbcdc557700 1 -- 172.21.15.196:0/3663632151 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcb0002210 con 0x7fbcb4004410 2020-05-18T21:40:29.117 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.115+0000 7fbcdaaf4700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7fbccc007ce0 tx=0x7fbccc00cc90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.117 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.115+0000 7fbccb7fe700 1 -- 172.21.15.196:0/3663632151 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13242 (secure 0 0 0) 0x7fbccc00b0b0 con 0x7fbcb4004410 2020-05-18T21:40:29.118 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:29.118 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":17,"fsid":"d048673a-3d91-4cc6-882d-e9cc70e7cd6c","created":"2020-05-18T21:40:15.911738+0000","modified":"2020-05-18T21:40:28.853113+0000","last_up_change":"2020-05-18T21:40:22.762927+0000","last_in_change":"2020-05-18T21:40:22.762927+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-05-18T21:40:19.808941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-05-18T21:40:26.944311+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"8db63a0a-61dc-42b5-9559-fddcf436cab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":12771},{"type":"v1","addr":"172.21.15.196:6801","nonce":12771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":12771},{"type":"v1","addr":"172.21.15.196:6803","nonce":12771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":12771},{"type":"v1","addr":"172.21.15.196:6807","nonce":12771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":12771},{"type":"v1","addr":"172.21.15.196:6805","nonce":12771}]},"public_addr":"172.21.15.196:6801/12771","cluster_addr":"172.21.15.196:6803/12771","heartbeat_back_addr":"172.21.15.196:6807/12771","heartbeat_front_addr":"172.21.15.196:6805/12771","state":["exists","up"]},{"osd":1,"uuid":"6773b0c3-9f49-42a2-b75b-c10c734ff086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":12772},{"type":"v1","addr":"172.21.15.196:6817","nonce":12772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":12772},{"type":"v1","addr":"172.21.15.196:6819","nonce":12772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":12772},{"type":"v1","addr":"172.21.15.196:6823","nonce":12772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":12772},{"type":"v1","addr":"172.21.15.196:6821","nonce":12772}]},"public_addr":"172.21.15.196:6817/12772","cluster_addr":"172.21.15.196:6819/12772","heartbeat_back_addr":"172.21.15.196:6823/12772","heartbeat_front_addr":"172.21.15.196:6821/12772","state":["exists","up"]},{"osd":2,"uuid":"ac1f2e05-9bb7-4768-b712-10e4d2ea4154","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":12773},{"type":"v1","addr":"172.21.15.196:6809","nonce":12773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":12773},{"type":"v1","addr":"172.21.15.196:6811","nonce":12773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":12773},{"type":"v1","addr":"172.21.15.196:6815","nonce":12773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":12773},{"type":"v1","addr":"172.21.15.196:6813","nonce":12773}]},"public_addr":"172.21.15.196:6809/12773","cluster_addr":"172.21.15.196:6811/12773","heartbeat_back_addr":"172.21.15.196:6815/12773","heartbeat_front_addr":"172.21.15.196:6813/12773","state":["exists","up"]},{"osd":3,"uuid":"bae5ab40-235b-4e0f-8d0d-524afd099b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":12775},{"type":"v1","addr":"172.21.15.196:6825","nonce":12775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":12775},{"type":"v1","addr":"172.21.15.196:6827","nonce":12775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":12775},{"type":"v1","addr":"172.21.15.196:6831","nonce":12775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":12775},{"type":"v1","addr":"172.21.15.196:6829","nonce":12775}]},"public_addr":"172.21.15.196:6825/12775","cluster_addr":"172.21.15.196:6827/12775","heartbeat_back_addr":"172.21.15.196:6831/12775","heartbeat_front_addr":"172.21.15.196:6829/12775","state":["exists","up"]},{"osd":4,"uuid":"275fc52c-58e4-4f35-a0cb-e117fe36bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":13040},{"type":"v1","addr":"172.21.15.80:6819","nonce":13040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6821","nonce":13040},{"type":"v1","addr":"172.21.15.80:6823","nonce":13040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6829","nonce":13040},{"type":"v1","addr":"172.21.15.80:6831","nonce":13040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":13040},{"type":"v1","addr":"172.21.15.80:6827","nonce":13040}]},"public_addr":"172.21.15.80:6819/13040","cluster_addr":"172.21.15.80:6823/13040","heartbeat_back_addr":"172.21.15.80:6831/13040","heartbeat_front_addr":"172.21.15.80:6827/13040","state":["exists","up"]},{"osd":5,"uuid":"9795b89c-e2f8-4abf-b9c7-796727a57002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":13037},{"type":"v1","addr":"172.21.15.80:6818","nonce":13037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":13037},{"type":"v1","addr":"172.21.15.80:6822","nonce":13037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":13037},{"type":"v1","addr":"172.21.15.80:6830","nonce":13037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":13037},{"type":"v1","addr":"172.21.15.80:6826","nonce":13037}]},"public_addr":"172.21.15.80:6818/13037","cluster_addr":"172.21.15.80:6822/13037","heartbeat_back_addr":"172.21.15.80:6830/13037","heartbeat_front_addr":"172.21.15.80:6826/13037","state":["exists","up"]},{"osd":6,"uuid":"e42df2e8-9cc5-4c7e-a243-bd18ccfbd002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":13039},{"type":"v1","addr":"172.21.15.80:6804","nonce":13039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":13039},{"type":"v1","addr":"172.21.15.80:6808","nonce":13039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":13039},{"type":"v1","addr":"172.21.15.80:6813","nonce":13039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":13039},{"type":"v1","addr":"172.21.15.80:6810","nonce":13039}]},"public_addr":"172.21.15.80:6804/13039","cluster_addr":"172.21.15.80:6808/13039","heartbeat_back_addr":"172.21.15.80:6813/13039","heartbeat_front_addr":"172.21.15.80:6810/13039","state":["exists","up"]},{"osd":7,"uuid":"ebd6c938-beff-4668-b70e-076ed22bb8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":13038},{"type":"v1","addr":"172.21.15.80:6801","nonce":13038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":13038},{"type":"v1","addr":"172.21.15.80:6805","nonce":13038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":13038},{"type":"v1","addr":"172.21.15.80:6817","nonce":13038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":13038},{"type":"v1","addr":"172.21.15.80:6811","nonce":13038}]},"public_addr":"172.21.15.80:6801/13038","cluster_addr":"172.21.15.80:6805/13038","heartbeat_back_addr":"172.21.15.80:6817/13038","heartbeat_front_addr":"172.21.15.80:6811/13038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.134515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.439836+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.476825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.176642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.452301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.427183+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.230250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.210168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-18T21:40:29.119 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.119+0000 7fbcdc557700 1 -- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbcb4058300 msgr2=0x7fbcb405a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.119+0000 7fbcdc557700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbcb4058300 0x7fbcb405a7c0 secure :-1 s=READY pgs=28 cs=0 l=1 rx=0x7fbcd0007ac0 tx=0x7fbcd000d570).stop 2020-05-18T21:40:29.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.119+0000 7fbcdc557700 1 -- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 msgr2=0x7fbcb400b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.119+0000 7fbcdaaf4700 1 -- 172.21.15.196:0/3663632151 reap_dead start 2020-05-18T21:40:29.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.119+0000 7fbcdc557700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 secure :-1 s=READY pgs=69 cs=0 l=1 rx=0x7fbccc007ce0 tx=0x7fbccc00cc90).stop 2020-05-18T21:40:29.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.119+0000 7fbcdc557700 1 -- 172.21.15.196:0/3663632151 shutdown_connections 2020-05-18T21:40:29.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.119+0000 7fbcdc557700 1 --2- 172.21.15.196:0/3663632151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcb4004410 0x7fbcb400b760 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.119+0000 7fbcdc557700 1 -- 172.21.15.196:0/3663632151 >> 172.21.15.196:0/3663632151 conn(0x7fbcd4069ac0 msgr2=0x7fbcd405eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:29.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.119+0000 7fbcdc557700 1 -- 172.21.15.196:0/3663632151 shutdown_connections 2020-05-18T21:40:29.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.119+0000 7fbcdc557700 1 -- 172.21.15.196:0/3663632151 wait complete. 2020-05-18T21:40:29.134 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-05-18T21:40:19.808941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-05-18T21:40:26.944311+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-05-18T21:40:29.135 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-05-18T21:40:29.236 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.235+0000 7f368b59c700 1 Processor -- start 2020-05-18T21:40:29.236 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.235+0000 7f368b59c700 1 -- start start 2020-05-18T21:40:29.239 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.235+0000 7f368b59c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3684069e00 0x7f368406a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:29.239 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.235+0000 7f368b59c700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3684137fd0 0x7f3684131990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:29.239 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.235+0000 7f368b59c700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36840a65b0 con 0x7f3684137b60 2020-05-18T21:40:29.239 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.235+0000 7f368b59c700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368406ac40 con 0x7f3684069e00 2020-05-18T21:40:29.240 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.235+0000 7f368b59c700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36840a6850 con 0x7f3684137fd0 2020-05-18T21:40:29.240 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.235+0000 7f368a59a700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3684069e00 0x7f368406a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.240 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.235+0000 7f368a59a700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3684069e00 0x7f368406a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34168/0 (socket says 172.21.15.196:34168) 2020-05-18T21:40:29.240 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.235+0000 7f368a59a700 1 -- 172.21.15.196:0/1386244542 learned_addr learned my addr 172.21.15.196:0/1386244542 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:29.240 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.235+0000 7f368ad9b700 1 --2- 172.21.15.196:0/1386244542 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3684137fd0 0x7f3684131990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.241 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f3689598700 1 -- 172.21.15.196:0/1386244542 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688944408 0 0) 0x7f36840a65b0 con 0x7f3684137b60 2020-05-18T21:40:29.241 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368a59a700 1 -- 172.21.15.196:0/1386244542 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3684137fd0 msgr2=0x7f3684131990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:29.241 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368a59a700 1 --2- 172.21.15.196:0/1386244542 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3684137fd0 0x7f3684131990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:29.241 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368a59a700 1 -- 172.21.15.196:0/1386244542 >> v1:172.21.15.80:6789/0 conn(0x7f3684137b60 legacy=0x7f3684131280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.241 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368a59a700 1 -- 172.21.15.196:0/1386244542 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36840a6af0 con 0x7f3684069e00 2020-05-18T21:40:29.242 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368a59a700 1 --2- 172.21.15.196:0/1386244542 >> v2:172.21.15.196:3300/0 conn(0x7f3684069e00 0x7f368406a7e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f3680002f70 tx=0x7f3680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.242 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368a59a700 1 --2- 172.21.15.196:0/1386244542 >> v2:172.21.15.196:3300/0 conn(0x7f3684069e00 0x7f368406a7e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f3680002f70 tx=0x7f3680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.242 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368a59a700 1 --2- 172.21.15.196:0/1386244542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684069e00 0x7f368406a7e0 secure :-1 s=READY pgs=70 cs=0 l=1 rx=0x7f3680002f70 tx=0x7f3680007a80).ready entity=mon.1 client_cookie=2a9620d4a232e9e8 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:29.242 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368a59a700 1 --2- 172.21.15.196:0/1386244542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684069e00 0x7f368406a7e0 secure :-1 s=THROTTLE_DONE pgs=70 cs=0 l=1 rx=0x7f3680002f70 tx=0x7f3680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.242 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368a59a700 1 --2- 172.21.15.196:0/1386244542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684069e00 0x7f368406a7e0 secure :-1 s=THROTTLE_DONE pgs=70 cs=0 l=1 rx=0x7f3680002f70 tx=0x7f3680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.243 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f3689598700 1 -- 172.21.15.196:0/1386244542 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36800116b0 con 0x7f3684069e00 2020-05-18T21:40:29.243 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368a59a700 1 --2- 172.21.15.196:0/1386244542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684069e00 0x7f368406a7e0 secure :-1 s=THROTTLE_DONE pgs=70 cs=0 l=1 rx=0x7f3680002f70 tx=0x7f3680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.243 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f3689598700 1 -- 172.21.15.196:0/1386244542 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3680011810 con 0x7f3684069e00 2020-05-18T21:40:29.243 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f3689598700 1 -- 172.21.15.196:0/1386244542 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3680011bf0 con 0x7f3684069e00 2020-05-18T21:40:29.243 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368b59c700 1 -- 172.21.15.196:0/1386244542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684069e00 msgr2=0x7f368406a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.243 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368b59c700 1 --2- 172.21.15.196:0/1386244542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684069e00 0x7f368406a7e0 secure :-1 s=READY pgs=70 cs=0 l=1 rx=0x7f3680002f70 tx=0x7f3680007a80).stop 2020-05-18T21:40:29.247 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368b59c700 1 -- 172.21.15.196:0/1386244542 shutdown_connections 2020-05-18T21:40:29.247 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368b59c700 1 --2- 172.21.15.196:0/1386244542 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3684137fd0 0x7f3684131990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:29.247 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368b59c700 1 --2- 172.21.15.196:0/1386244542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684069e00 0x7f368406a7e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.248 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368b59c700 1 -- 172.21.15.196:0/1386244542 >> 172.21.15.196:0/1386244542 conn(0x7f368409d850 msgr2=0x7f36840aa440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:29.248 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368b59c700 1 -- 172.21.15.196:0/1386244542 shutdown_connections 2020-05-18T21:40:29.248 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368b59c700 1 -- 172.21.15.196:0/1386244542 wait complete. 2020-05-18T21:40:29.248 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368b59c700 1 Processor -- start 2020-05-18T21:40:29.248 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.239+0000 7f368b59c700 1 -- start start 2020-05-18T21:40:29.249 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368b59c700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3684069e00 0x7f368412b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.249 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368b59c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3684137fd0 0x7f3684137130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.249 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368b59c700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3684136270 con 0x7f3684137b60 2020-05-18T21:40:29.249 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368b59c700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36841376b0 con 0x7f3684069e00 2020-05-18T21:40:29.249 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368b59c700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368409f440 con 0x7f3684137fd0 2020-05-18T21:40:29.250 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368ad9b700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3684137fd0 0x7f3684137130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.250 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f3689d99700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f3684137b60 0x7f3684135b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42932/0 (socket says 172.21.15.196:42932) 2020-05-18T21:40:29.250 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f3689d99700 1 -- 172.21.15.196:0/1142891457 learned_addr learned my addr 172.21.15.196:0/1142891457 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:29.250 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368a59a700 1 --2- 172.21.15.196:0/1142891457 >> v2:172.21.15.80:3300/0 conn(0x7f3684069e00 0x7f368412b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.250 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368ad9b700 1 -- 172.21.15.196:0/1142891457 >> v2:172.21.15.80:3300/0 conn(0x7f3684069e00 msgr2=0x7f368412b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.251 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368ad9b700 1 --2- 172.21.15.196:0/1142891457 >> v2:172.21.15.80:3300/0 conn(0x7f3684069e00 0x7f368412b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.251 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368ad9b700 1 -- 172.21.15.196:0/1142891457 >> v1:172.21.15.196:6789/0 conn(0x7f3684137b60 legacy=0x7f3684135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.251 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f36777fe700 1 -- 172.21.15.196:0/1142891457 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869321554 0 0) 0x7f3684136270 con 0x7f3684137b60 2020-05-18T21:40:29.251 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368ad9b700 1 -- 172.21.15.196:0/1142891457 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368409f6e0 con 0x7f3684137fd0 2020-05-18T21:40:29.251 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368ad9b700 1 --2- 172.21.15.196:0/1142891457 >> v2:172.21.15.196:3300/0 conn(0x7f3684137fd0 0x7f3684137130 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f367c0032f0 tx=0x7f367c010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.252 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368ad9b700 1 --2- 172.21.15.196:0/1142891457 >> v2:172.21.15.196:3300/0 conn(0x7f3684137fd0 0x7f3684137130 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f367c0032f0 tx=0x7f367c010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.252 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368ad9b700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684137fd0 0x7f3684137130 secure :-1 s=READY pgs=72 cs=0 l=1 rx=0x7f367c0032f0 tx=0x7f367c010f00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:29.252 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368ad9b700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684137fd0 0x7f3684137130 secure :-1 s=THROTTLE_DONE pgs=72 cs=0 l=1 rx=0x7f367c0032f0 tx=0x7f367c010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.252 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f36777fe700 1 -- 172.21.15.196:0/1142891457 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f367c005a80 con 0x7f3684137fd0 2020-05-18T21:40:29.252 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368ad9b700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684137fd0 0x7f3684137130 secure :-1 s=THROTTLE_DONE pgs=72 cs=0 l=1 rx=0x7f367c0032f0 tx=0x7f367c010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.252 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368ad9b700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684137fd0 0x7f3684137130 secure :-1 s=THROTTLE_DONE pgs=72 cs=0 l=1 rx=0x7f367c0032f0 tx=0x7f367c010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.253 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f36777fe700 1 -- 172.21.15.196:0/1142891457 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c005be0 con 0x7f3684137fd0 2020-05-18T21:40:29.253 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368b59c700 1 -- 172.21.15.196:0/1142891457 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36841356a0 con 0x7f3684137fd0 2020-05-18T21:40:29.253 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f36777fe700 1 -- 172.21.15.196:0/1142891457 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f367c012950 con 0x7f3684137fd0 2020-05-18T21:40:29.253 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368b59c700 1 -- 172.21.15.196:0/1142891457 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368409fb80 con 0x7f3684137fd0 2020-05-18T21:40:29.254 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.243+0000 7f368ad9b700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684137fd0 0x7f3684137130 secure :-1 s=THROTTLE_DONE pgs=72 cs=0 l=1 rx=0x7f367c0032f0 tx=0x7f367c010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.254 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.247+0000 7f368b59c700 1 -- 172.21.15.196:0/1142891457 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36840569e0 con 0x7f3684137fd0 2020-05-18T21:40:29.254 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.247+0000 7f36777fe700 1 -- 172.21.15.196:0/1142891457 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f367c012b80 con 0x7f3684137fd0 2020-05-18T21:40:29.254 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.247+0000 7f368ad9b700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684137fd0 0x7f3684137130 secure :-1 s=THROTTLE_DONE pgs=72 cs=0 l=1 rx=0x7f367c0032f0 tx=0x7f367c010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.254 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.247+0000 7f36777fe700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f366404f7d0 0x7f3664051c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.255 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.247+0000 7f36777fe700 1 -- 172.21.15.196:0/1142891457 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f367c063bf0 con 0x7f3684137fd0 2020-05-18T21:40:29.255 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.247+0000 7f368a59a700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f366404f7d0 0x7f3664051c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.255 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.247+0000 7f368a59a700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f366404f7d0 0x7f3664051c90 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f3680002f70 tx=0x7f3680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.255 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.247+0000 7f368a59a700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f366404f7d0 0x7f3664051c90 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f3680002f70 tx=0x7f3680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.255 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.247+0000 7f368a59a700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f366404f7d0 0x7f3664051c90 secure :-1 s=READY pgs=29 cs=0 l=1 rx=0x7f3680002f70 tx=0x7f3680007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:29.256 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.251+0000 7f368ad9b700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684137fd0 0x7f3684137130 secure :-1 s=THROTTLE_DONE pgs=72 cs=0 l=1 rx=0x7f367c0032f0 tx=0x7f367c010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.256 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.251+0000 7f36777fe700 1 -- 172.21.15.196:0/1142891457 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f367c03df20 con 0x7f3684137fd0 2020-05-18T21:40:29.365 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f368b59c700 1 -- 172.21.15.196:0/1142891457 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3684067250 con 0x7f3684137fd0 2020-05-18T21:40:29.366 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f368ad9b700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684137fd0 0x7f3684137130 secure :-1 s=THROTTLE_DONE pgs=72 cs=0 l=1 rx=0x7f367c0032f0 tx=0x7f367c010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.367 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f36777fe700 1 -- 172.21.15.196:0/1142891457 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7f367c041540 con 0x7f3684137fd0 2020-05-18T21:40:29.367 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2020-05-18T21:40:29.368 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f368b59c700 1 -- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f366404f7d0 msgr2=0x7f3664051c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.368 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f368b59c700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f366404f7d0 0x7f3664051c90 secure :-1 s=READY pgs=29 cs=0 l=1 rx=0x7f3680002f70 tx=0x7f3680007a80).stop 2020-05-18T21:40:29.369 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f368b59c700 1 -- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684137fd0 msgr2=0x7f3684137130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.369 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f368b59c700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684137fd0 0x7f3684137130 secure :-1 s=READY pgs=72 cs=0 l=1 rx=0x7f367c0032f0 tx=0x7f367c010f00).stop 2020-05-18T21:40:29.369 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f368b59c700 1 -- 172.21.15.196:0/1142891457 shutdown_connections 2020-05-18T21:40:29.369 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f368b59c700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f366404f7d0 0x7f3664051c90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.369 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f368b59c700 1 --2- 172.21.15.196:0/1142891457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3684137fd0 0x7f3684137130 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.370 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f368b59c700 1 --2- 172.21.15.196:0/1142891457 >> v2:172.21.15.80:3300/0 conn(0x7f3684069e00 0x7f368412b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.370 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f368b59c700 1 -- 172.21.15.196:0/1142891457 >> 172.21.15.196:0/1142891457 conn(0x7f368409d850 msgr2=0x7f36840a3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:29.370 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f368b59c700 1 -- 172.21.15.196:0/1142891457 shutdown_connections 2020-05-18T21:40:29.370 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.367+0000 7f368b59c700 1 -- 172.21.15.196:0/1142891457 wait complete. 2020-05-18T21:40:29.379 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-05-18T21:40:29.488 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.483+0000 7f7b524bd700 1 Processor -- start 2020-05-18T21:40:29.489 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.483+0000 7f7b524bd700 1 -- start start 2020-05-18T21:40:29.489 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.483+0000 7f7b524bd700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7b4c0a91e0 0x7f7b4c13b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:29.489 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.483+0000 7f7b524bd700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c05b670 con 0x7f7b4c066680 2020-05-18T21:40:29.490 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.483+0000 7f7b524bd700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c0a9760 con 0x7f7b4c05c1b0 2020-05-18T21:40:29.490 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.483+0000 7f7b524bd700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b4c13b630 con 0x7f7b4c0a91e0 2020-05-18T21:40:29.490 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.483+0000 7f7b50cba700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f7b4c05c1b0 0x7f7b4c13aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42940/0 (socket says 172.21.15.196:42940) 2020-05-18T21:40:29.490 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.483+0000 7f7b50cba700 1 -- 172.21.15.196:0/4163191849 learned_addr learned my addr 172.21.15.196:0/4163191849 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:29.490 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.483+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/4163191849 >> v2:172.21.15.80:3300/0 conn(0x7f7b4c0a91e0 0x7f7b4c13b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.491 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811469261 0 0) 0x7f7b4c0a9760 con 0x7f7b4c05c1b0 2020-05-18T21:40:29.491 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b34003270 con 0x7f7b4c05c1b0 2020-05-18T21:40:29.491 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134009533 0 0) 0x7f7b4c05b670 con 0x7f7b4c066680 2020-05-18T21:40:29.491 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b4c0a9760 con 0x7f7b4c066680 2020-05-18T21:40:29.491 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4021448379 0 0) 0x7f7b34003270 con 0x7f7b4c05c1b0 2020-05-18T21:40:29.492 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 >> v2:172.21.15.80:3300/0 conn(0x7f7b4c0a91e0 msgr2=0x7f7b4c13b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:29.492 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 --2- 172.21.15.196:0/4163191849 >> v2:172.21.15.80:3300/0 conn(0x7f7b4c0a91e0 0x7f7b4c13b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:29.492 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 >> v1:172.21.15.80:6789/0 conn(0x7f7b4c066680 legacy=0x7f7b4c05baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.492 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c0a9a00 con 0x7f7b4c05c1b0 2020-05-18T21:40:29.492 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7f7b48001890 con 0x7f7b4c05c1b0 2020-05-18T21:40:29.492 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 >> v1:172.21.15.196:6789/0 conn(0x7f7b4c05c1b0 legacy=0x7f7b4c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.493 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b340038f0 0x7f7b34003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:29.493 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b34004510 0x7f7b34004990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:29.493 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b34004df0 0x7f7b3400b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:29.493 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c13b630 con 0x7f7b340038f0 2020-05-18T21:40:29.493 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340053e0 con 0x7f7b34004510 2020-05-18T21:40:29.494 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3400bc40 con 0x7f7b34004df0 2020-05-18T21:40:29.494 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/4163191849 >> v2:172.21.15.80:3300/0 conn(0x7f7b4c0a91e0 0x7f7b4c13b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:29.494 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b50cba700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b34004510 0x7f7b34004990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.494 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b514bb700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b340038f0 0x7f7b34003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.494 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b34004df0 0x7f7b3400b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.495 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b50cba700 1 -- 172.21.15.196:0/4163191849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b34004df0 msgr2=0x7f7b3400b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:29.495 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b50cba700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b34004df0 0x7f7b3400b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:29.495 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b50cba700 1 -- 172.21.15.196:0/4163191849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b340038f0 msgr2=0x7f7b34003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:29.495 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b50cba700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b340038f0 0x7f7b34003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:29.495 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b50cba700 1 -- 172.21.15.196:0/4163191849 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b34003270 con 0x7f7b34004510 2020-05-18T21:40:29.496 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b34004df0 0x7f7b3400b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request state changed! 2020-05-18T21:40:29.496 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b51cbc700 1 -- 172.21.15.196:0/4163191849 reap_dead start 2020-05-18T21:40:29.496 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b50cba700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b34004510 0x7f7b34004990 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f7b48001660 tx=0x7f7b4c05e6a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.496 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b50cba700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b34004510 0x7f7b34004990 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f7b48001660 tx=0x7f7b4c05e6a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.496 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b50cba700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b34004510 0x7f7b34004990 secure :-1 s=READY pgs=74 cs=0 l=1 rx=0x7f7b48001660 tx=0x7f7b4c05e6a0).ready entity=mon.1 client_cookie=6d9bceb769d36038 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:29.497 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b50cba700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b34004510 0x7f7b34004990 secure :-1 s=THROTTLE_DONE pgs=74 cs=0 l=1 rx=0x7f7b48001660 tx=0x7f7b4c05e6a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.497 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b4800b460 con 0x7f7b34004510 2020-05-18T21:40:29.497 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b50cba700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b34004510 0x7f7b34004990 secure :-1 s=THROTTLE_DONE pgs=74 cs=0 l=1 rx=0x7f7b48001660 tx=0x7f7b4c05e6a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.497 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b50cba700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b34004510 0x7f7b34004990 secure :-1 s=THROTTLE_DONE pgs=74 cs=0 l=1 rx=0x7f7b48001660 tx=0x7f7b4c05e6a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.497 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b480101c0 con 0x7f7b34004510 2020-05-18T21:40:29.498 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b3bfff700 1 -- 172.21.15.196:0/4163191849 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b48010520 con 0x7f7b34004510 2020-05-18T21:40:29.498 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b524bd700 1 -- 172.21.15.196:0/4163191849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b34004510 msgr2=0x7f7b34004990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.498 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b524bd700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b34004510 0x7f7b34004990 secure :-1 s=READY pgs=74 cs=0 l=1 rx=0x7f7b48001660 tx=0x7f7b4c05e6a0).stop 2020-05-18T21:40:29.498 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b524bd700 1 -- 172.21.15.196:0/4163191849 shutdown_connections 2020-05-18T21:40:29.498 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b524bd700 1 --2- 172.21.15.196:0/4163191849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b34004510 0x7f7b34004990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.499 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b524bd700 1 -- 172.21.15.196:0/4163191849 >> 172.21.15.196:0/4163191849 conn(0x7f7b4c069ac0 msgr2=0x7f7b4c065c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:29.499 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b524bd700 1 -- 172.21.15.196:0/4163191849 shutdown_connections 2020-05-18T21:40:29.499 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.487+0000 7f7b524bd700 1 -- 172.21.15.196:0/4163191849 wait complete. 2020-05-18T21:40:29.503 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.491+0000 7f7b524bd700 1 Processor -- start 2020-05-18T21:40:29.504 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.491+0000 7f7b524bd700 1 -- start start 2020-05-18T21:40:29.504 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.491+0000 7f7b524bd700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7b4c0a5520 0x7f7b4c09e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.504 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.491+0000 7f7b524bd700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c0aea50 con 0x7f7b34004510 2020-05-18T21:40:29.504 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.491+0000 7f7b524bd700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c05a9d0 con 0x7f7b4c145b30 2020-05-18T21:40:29.505 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.491+0000 7f7b524bd700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b4c09ec30 con 0x7f7b4c0a5520 2020-05-18T21:40:29.505 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.491+0000 7f7b51cbc700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f7b4c145b30 0x7f7b4c145f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42954/0 (socket says 172.21.15.196:42954) 2020-05-18T21:40:29.505 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.491+0000 7f7b51cbc700 1 -- 172.21.15.196:0/239922413 learned_addr learned my addr 172.21.15.196:0/239922413 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:29.505 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.491+0000 7f7b50cba700 1 --2- 172.21.15.196:0/239922413 >> v2:172.21.15.80:3300/0 conn(0x7f7b4c0a5520 0x7f7b4c09e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.505 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.491+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003309001 0 0) 0x7f7b4c05a9d0 con 0x7f7b4c145b30 2020-05-18T21:40:29.506 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.491+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b2c002fe0 con 0x7f7b4c145b30 2020-05-18T21:40:29.506 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.491+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642688430 0 0) 0x7f7b4c0aea50 con 0x7f7b34004510 2020-05-18T21:40:29.506 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.491+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b4c05a9d0 con 0x7f7b34004510 2020-05-18T21:40:29.506 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1005370841 0 0) 0x7f7b2c002fe0 con 0x7f7b4c145b30 2020-05-18T21:40:29.506 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 --> v1:172.21.15.196:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7b4c0aea50 con 0x7f7b4c145b30 2020-05-18T21:40:29.507 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7f7b44001660 con 0x7f7b4c145b30 2020-05-18T21:40:29.508 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 >> v2:172.21.15.80:3300/0 conn(0x7f7b4c0a5520 msgr2=0x7f7b4c09e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.508 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 --2- 172.21.15.196:0/239922413 >> v2:172.21.15.80:3300/0 conn(0x7f7b4c0a5520 0x7f7b4c09e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.508 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 >> v1:172.21.15.196:6789/0 conn(0x7f7b4c145b30 legacy=0x7f7b4c145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.508 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 >> v1:172.21.15.80:6789/0 conn(0x7f7b34004510 legacy=0x7f7b4c09ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.510 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b2c004bf0 0x7f7b2c004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.510 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b2c003b30 0x7f7b2c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.510 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.511 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c09ec30 con 0x7f7b2c004bf0 2020-05-18T21:40:29.511 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c00bbc0 con 0x7f7b2c0043d0 2020-05-18T21:40:29.511 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c00bd00 con 0x7f7b2c003b30 2020-05-18T21:40:29.511 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b50cba700 1 --2- 172.21.15.196:0/239922413 >> v2:172.21.15.80:3300/0 conn(0x7f7b4c0a5520 0x7f7b4c09e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:29.511 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.512 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b514bb700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b2c004bf0 0x7f7b2c004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.512 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b50cba700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b2c003b30 0x7f7b2c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.512 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 -- 172.21.15.196:0/239922413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b2c003b30 msgr2=0x7f7b2c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.512 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b2c003b30 0x7f7b2c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.513 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 -- 172.21.15.196:0/239922413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b2c004bf0 msgr2=0x7f7b2c004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.513 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b2c004bf0 0x7f7b2c004ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.513 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 -- 172.21.15.196:0/239922413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c1381f0 con 0x7f7b2c0043d0 2020-05-18T21:40:29.513 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b524bd700 1 -- 172.21.15.196:0/239922413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4c09e170 con 0x7f7b2c0043d0 2020-05-18T21:40:29.513 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7b440032f0 tx=0x7f7b440115d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.514 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b524bd700 1 -- 172.21.15.196:0/239922413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4c0a0660 con 0x7f7b2c0043d0 2020-05-18T21:40:29.514 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 -- 172.21.15.196:0/239922413 reap_dead start 2020-05-18T21:40:29.514 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7b440032f0 tx=0x7f7b440115d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.514 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 secure :-1 s=READY pgs=76 cs=0 l=1 rx=0x7f7b440032f0 tx=0x7f7b440115d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:29.515 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 secure :-1 s=THROTTLE_DONE pgs=76 cs=0 l=1 rx=0x7f7b440032f0 tx=0x7f7b440115d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.515 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b44005e70 con 0x7f7b2c0043d0 2020-05-18T21:40:29.515 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 secure :-1 s=THROTTLE_DONE pgs=76 cs=0 l=1 rx=0x7f7b440032f0 tx=0x7f7b440115d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.515 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b44007560 con 0x7f7b2c0043d0 2020-05-18T21:40:29.515 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 secure :-1 s=THROTTLE_DONE pgs=76 cs=0 l=1 rx=0x7f7b440032f0 tx=0x7f7b440115d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.516 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b440078c0 con 0x7f7b2c0043d0 2020-05-18T21:40:29.516 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b524bd700 1 -- 172.21.15.196:0/239922413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b28003380 con 0x7f7b2c0043d0 2020-05-18T21:40:29.516 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 secure :-1 s=THROTTLE_DONE pgs=76 cs=0 l=1 rx=0x7f7b440032f0 tx=0x7f7b440115d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.516 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f7b4400b1a0 con 0x7f7b2c0043d0 2020-05-18T21:40:29.516 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 secure :-1 s=THROTTLE_DONE pgs=76 cs=0 l=1 rx=0x7f7b440032f0 tx=0x7f7b440115d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.516 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f7b2c053260 0x7f7b2c055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.517 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7b440645e0 con 0x7f7b2c0043d0 2020-05-18T21:40:29.517 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.495+0000 7f7b514bb700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f7b2c053260 0x7f7b2c055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.517 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.499+0000 7f7b514bb700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f7b2c053260 0x7f7b2c055720 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7b3c001c80 tx=0x7f7b4c05e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.517 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.499+0000 7f7b514bb700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f7b2c053260 0x7f7b2c055720 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7b3c001c80 tx=0x7f7b4c05e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.517 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.499+0000 7f7b514bb700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f7b2c053260 0x7f7b2c055720 secure :-1 s=READY pgs=30 cs=0 l=1 rx=0x7f7b3c001c80 tx=0x7f7b4c05e380).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:29.518 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.503+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 secure :-1 s=THROTTLE_DONE pgs=76 cs=0 l=1 rx=0x7f7b440032f0 tx=0x7f7b440115d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.518 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.503+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f7b4403e910 con 0x7f7b2c0043d0 2020-05-18T21:40:29.617 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.615+0000 7f7b524bd700 1 -- 172.21.15.196:0/239922413 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7b28002240 con 0x7f7b2c0043d0 2020-05-18T21:40:29.618 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.619+0000 7f7b51cbc700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 secure :-1 s=THROTTLE_DONE pgs=76 cs=0 l=1 rx=0x7f7b440032f0 tx=0x7f7b440115d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.618 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.619+0000 7f7b3a7fc700 1 -- 172.21.15.196:0/239922413 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7b44041f30 con 0x7f7b2c0043d0 2020-05-18T21:40:29.618 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 8 2020-05-18T21:40:29.619 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.619+0000 7f7b524bd700 1 -- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f7b2c053260 msgr2=0x7f7b2c055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.620 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.619+0000 7f7b524bd700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f7b2c053260 0x7f7b2c055720 secure :-1 s=READY pgs=30 cs=0 l=1 rx=0x7f7b3c001c80 tx=0x7f7b4c05e380).stop 2020-05-18T21:40:29.620 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.619+0000 7f7b524bd700 1 -- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 msgr2=0x7f7b2c00b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.620 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.619+0000 7f7b524bd700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 secure :-1 s=READY pgs=76 cs=0 l=1 rx=0x7f7b440032f0 tx=0x7f7b440115d0).stop 2020-05-18T21:40:29.620 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.619+0000 7f7b524bd700 1 -- 172.21.15.196:0/239922413 shutdown_connections 2020-05-18T21:40:29.620 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.619+0000 7f7b524bd700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f7b2c053260 0x7f7b2c055720 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.621 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.619+0000 7f7b524bd700 1 --2- 172.21.15.196:0/239922413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b2c0043d0 0x7f7b2c00b760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.621 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.619+0000 7f7b524bd700 1 -- 172.21.15.196:0/239922413 >> 172.21.15.196:0/239922413 conn(0x7f7b4c069ac0 msgr2=0x7f7b4c05eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:29.621 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.619+0000 7f7b524bd700 1 -- 172.21.15.196:0/239922413 shutdown_connections 2020-05-18T21:40:29.621 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.619+0000 7f7b524bd700 1 -- 172.21.15.196:0/239922413 wait complete. 2020-05-18T21:40:29.629 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-05-18T21:40:29.731 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda5ffff700 1 Processor -- start 2020-05-18T21:40:29.731 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda5ffff700 1 -- start start 2020-05-18T21:40:29.734 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda5ffff700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fda600a7220 0x7fda600a3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:29.734 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda5ffff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fda600a3dd0 0x7fda60140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:29.734 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda5ffff700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6009e0b0 con 0x7fda60140590 2020-05-18T21:40:29.735 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda5ffff700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda600a3580 con 0x7fda600a7220 2020-05-18T21:40:29.735 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda5ffff700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda600a20f0 con 0x7fda600a3dd0 2020-05-18T21:40:29.735 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda57fff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fda600a3dd0 0x7fda60140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.735 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda57fff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fda600a3dd0 0x7fda60140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34204/0 (socket says 172.21.15.196:34204) 2020-05-18T21:40:29.735 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda57fff700 1 -- 172.21.15.196:0/3258022496 learned_addr learned my addr 172.21.15.196:0/3258022496 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:29.736 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda5effd700 1 --2- 172.21.15.196:0/3258022496 >> v2:172.21.15.80:3300/0 conn(0x7fda600a7220 0x7fda600a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.736 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda57fff700 1 -- 172.21.15.196:0/3258022496 >> v2:172.21.15.80:3300/0 conn(0x7fda600a7220 msgr2=0x7fda600a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:29.736 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda57fff700 1 --2- 172.21.15.196:0/3258022496 >> v2:172.21.15.80:3300/0 conn(0x7fda600a7220 0x7fda600a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:29.736 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda57fff700 1 -- 172.21.15.196:0/3258022496 >> v1:172.21.15.196:6789/0 conn(0x7fda60140590 legacy=0x7fda600a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.737 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda57fff700 1 -- 172.21.15.196:0/3258022496 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda600a6a60 con 0x7fda600a3dd0 2020-05-18T21:40:29.737 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda57fff700 1 --2- 172.21.15.196:0/3258022496 >> v2:172.21.15.196:3300/0 conn(0x7fda600a3dd0 0x7fda60140130 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fda48002f70 tx=0x7fda48007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.737 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda57fff700 1 --2- 172.21.15.196:0/3258022496 >> v2:172.21.15.196:3300/0 conn(0x7fda600a3dd0 0x7fda60140130 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fda48002f70 tx=0x7fda48007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.737 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda57fff700 1 --2- 172.21.15.196:0/3258022496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda600a3dd0 0x7fda60140130 secure :-1 s=READY pgs=78 cs=0 l=1 rx=0x7fda48002f70 tx=0x7fda48007a80).ready entity=mon.1 client_cookie=c11c5a9ad49296bb server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:29.737 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda57fff700 1 --2- 172.21.15.196:0/3258022496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda600a3dd0 0x7fda60140130 secure :-1 s=THROTTLE_DONE pgs=78 cs=0 l=1 rx=0x7fda48002f70 tx=0x7fda48007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.738 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda57fff700 1 --2- 172.21.15.196:0/3258022496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda600a3dd0 0x7fda60140130 secure :-1 s=THROTTLE_DONE pgs=78 cs=0 l=1 rx=0x7fda48002f70 tx=0x7fda48007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.738 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda57fff700 1 --2- 172.21.15.196:0/3258022496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda600a3dd0 0x7fda60140130 secure :-1 s=THROTTLE_DONE pgs=78 cs=0 l=1 rx=0x7fda48002f70 tx=0x7fda48007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.738 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda5e7fc700 1 -- 172.21.15.196:0/3258022496 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda48011760 con 0x7fda600a3dd0 2020-05-18T21:40:29.739 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda5e7fc700 1 -- 172.21.15.196:0/3258022496 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda480118c0 con 0x7fda600a3dd0 2020-05-18T21:40:29.739 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda5e7fc700 1 -- 172.21.15.196:0/3258022496 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda48011ce0 con 0x7fda600a3dd0 2020-05-18T21:40:29.739 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda5ffff700 1 -- 172.21.15.196:0/3258022496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda600a3dd0 msgr2=0x7fda60140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.739 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.731+0000 7fda5ffff700 1 --2- 172.21.15.196:0/3258022496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda600a3dd0 0x7fda60140130 secure :-1 s=READY pgs=78 cs=0 l=1 rx=0x7fda48002f70 tx=0x7fda48007a80).stop 2020-05-18T21:40:29.739 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 -- 172.21.15.196:0/3258022496 shutdown_connections 2020-05-18T21:40:29.740 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 --2- 172.21.15.196:0/3258022496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda600a3dd0 0x7fda60140130 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.740 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 --2- 172.21.15.196:0/3258022496 >> v2:172.21.15.80:3300/0 conn(0x7fda600a7220 0x7fda600a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:29.740 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 -- 172.21.15.196:0/3258022496 >> 172.21.15.196:0/3258022496 conn(0x7fda6005a7c0 msgr2=0x7fda6005abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:29.740 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 -- 172.21.15.196:0/3258022496 shutdown_connections 2020-05-18T21:40:29.740 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 -- 172.21.15.196:0/3258022496 wait complete. 2020-05-18T21:40:29.744 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 Processor -- start 2020-05-18T21:40:29.744 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 -- start start 2020-05-18T21:40:29.744 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda600a3dd0 0x7fda6013db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.744 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fda600a7220 0x7fda600a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.745 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fda60140590 0x7fda6009b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.745 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6009d5d0 con 0x7fda600a7220 2020-05-18T21:40:29.745 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6009aae0 con 0x7fda60140590 2020-05-18T21:40:29.745 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6009d7c0 con 0x7fda600a3dd0 2020-05-18T21:40:29.746 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fda60140590 0x7fda6009b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.746 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fda60140590 0x7fda6009b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34212/0 (socket says 172.21.15.196:34212) 2020-05-18T21:40:29.746 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 -- 172.21.15.196:0/590803725 learned_addr learned my addr 172.21.15.196:0/590803725 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:29.746 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda57fff700 1 --2- 172.21.15.196:0/590803725 >> v2:172.21.15.80:3300/0 conn(0x7fda600a7220 0x7fda600a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.746 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5effd700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda600a3dd0 0x7fda6013db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.747 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 -- 172.21.15.196:0/590803725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda600a3dd0 msgr2=0x7fda6013db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.747 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda600a3dd0 0x7fda6013db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.747 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 -- 172.21.15.196:0/590803725 >> v2:172.21.15.80:3300/0 conn(0x7fda600a7220 msgr2=0x7fda600a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.747 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 --2- 172.21.15.196:0/590803725 >> v2:172.21.15.80:3300/0 conn(0x7fda600a7220 0x7fda600a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.747 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 -- 172.21.15.196:0/590803725 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6009da60 con 0x7fda60140590 2020-05-18T21:40:29.748 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5effd700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda600a3dd0 0x7fda6013db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request state changed! 2020-05-18T21:40:29.748 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 --2- 172.21.15.196:0/590803725 >> v2:172.21.15.196:3300/0 conn(0x7fda60140590 0x7fda6009b660 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fda600a3850 tx=0x7fda5800c850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.748 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 --2- 172.21.15.196:0/590803725 >> v2:172.21.15.196:3300/0 conn(0x7fda60140590 0x7fda6009b660 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fda600a3850 tx=0x7fda5800c850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.748 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda60140590 0x7fda6009b660 secure :-1 s=READY pgs=79 cs=0 l=1 rx=0x7fda600a3850 tx=0x7fda5800c850).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:29.748 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda60140590 0x7fda6009b660 secure :-1 s=THROTTLE_DONE pgs=79 cs=0 l=1 rx=0x7fda600a3850 tx=0x7fda5800c850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.749 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5cff9700 1 -- 172.21.15.196:0/590803725 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda580101e0 con 0x7fda60140590 2020-05-18T21:40:29.749 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda60140590 0x7fda6009b660 secure :-1 s=THROTTLE_DONE pgs=79 cs=0 l=1 rx=0x7fda600a3850 tx=0x7fda5800c850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.749 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5f7fe700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda60140590 0x7fda6009b660 secure :-1 s=THROTTLE_DONE pgs=79 cs=0 l=1 rx=0x7fda600a3850 tx=0x7fda5800c850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.749 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5cff9700 1 -- 172.21.15.196:0/590803725 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda58010340 con 0x7fda60140590 2020-05-18T21:40:29.749 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 -- 172.21.15.196:0/590803725 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda600a1060 con 0x7fda60140590 2020-05-18T21:40:29.750 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5cff9700 1 -- 172.21.15.196:0/590803725 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda58010760 con 0x7fda60140590 2020-05-18T21:40:29.750 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.735+0000 7fda5ffff700 1 -- 172.21.15.196:0/590803725 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6009bc60 con 0x7fda60140590 2020-05-18T21:40:29.750 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.739+0000 7fda5f7fe700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda60140590 0x7fda6009b660 secure :-1 s=THROTTLE_DONE pgs=79 cs=0 l=1 rx=0x7fda600a3850 tx=0x7fda5800c850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.750 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.739+0000 7fda5cff9700 1 -- 172.21.15.196:0/590803725 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fda580068a0 con 0x7fda60140590 2020-05-18T21:40:29.750 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.739+0000 7fda5f7fe700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda60140590 0x7fda6009b660 secure :-1 s=THROTTLE_DONE pgs=79 cs=0 l=1 rx=0x7fda600a3850 tx=0x7fda5800c850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.751 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.739+0000 7fda5ffff700 1 -- 172.21.15.196:0/590803725 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda600569e0 con 0x7fda60140590 2020-05-18T21:40:29.751 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.739+0000 7fda5cff9700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fda4404f9e0 0x7fda44051ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.751 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.739+0000 7fda5cff9700 1 -- 172.21.15.196:0/590803725 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fda58063d30 con 0x7fda60140590 2020-05-18T21:40:29.751 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.739+0000 7fda5effd700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fda4404f9e0 0x7fda44051ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.751 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.739+0000 7fda5effd700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fda4404f9e0 0x7fda44051ea0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fda50002f70 tx=0x7fda50008de0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.752 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.739+0000 7fda5effd700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fda4404f9e0 0x7fda44051ea0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fda50002f70 tx=0x7fda50008de0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.752 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.739+0000 7fda5effd700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fda4404f9e0 0x7fda44051ea0 secure :-1 s=READY pgs=31 cs=0 l=1 rx=0x7fda50002f70 tx=0x7fda50008de0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:29.752 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.743+0000 7fda5f7fe700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda60140590 0x7fda6009b660 secure :-1 s=THROTTLE_DONE pgs=79 cs=0 l=1 rx=0x7fda600a3850 tx=0x7fda5800c850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.753 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.743+0000 7fda5cff9700 1 -- 172.21.15.196:0/590803725 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7fda5803e0c0 con 0x7fda60140590 2020-05-18T21:40:29.856 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.855+0000 7fda5ffff700 1 -- 172.21.15.196:0/590803725 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fda6009de70 con 0x7fda60140590 2020-05-18T21:40:29.857 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.855+0000 7fda5f7fe700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda60140590 0x7fda6009b660 secure :-1 s=THROTTLE_DONE pgs=79 cs=0 l=1 rx=0x7fda600a3850 tx=0x7fda5800c850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.857 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.855+0000 7fda5cff9700 1 -- 172.21.15.196:0/590803725 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+143 (secure 0 0 0) 0x7fda580416e0 con 0x7fda60140590 2020-05-18T21:40:29.857 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:29.858 INFO:teuthology.orchestra.run.smithi196.stdout:[ 2020-05-18T21:40:29.858 INFO:teuthology.orchestra.run.smithi196.stdout: { 2020-05-18T21:40:29.858 INFO:teuthology.orchestra.run.smithi196.stdout: "poolnum": 1, 2020-05-18T21:40:29.858 INFO:teuthology.orchestra.run.smithi196.stdout: "poolname": "device_health_metrics" 2020-05-18T21:40:29.858 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2020-05-18T21:40:29.859 INFO:teuthology.orchestra.run.smithi196.stdout: { 2020-05-18T21:40:29.859 INFO:teuthology.orchestra.run.smithi196.stdout: "poolnum": 2, 2020-05-18T21:40:29.859 INFO:teuthology.orchestra.run.smithi196.stdout: "poolname": "rbd" 2020-05-18T21:40:29.859 INFO:teuthology.orchestra.run.smithi196.stdout: } 2020-05-18T21:40:29.859 INFO:teuthology.orchestra.run.smithi196.stdout:] 2020-05-18T21:40:29.860 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.859+0000 7fda5ffff700 1 -- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fda4404f9e0 msgr2=0x7fda44051ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.861 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.859+0000 7fda5ffff700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fda4404f9e0 0x7fda44051ea0 secure :-1 s=READY pgs=31 cs=0 l=1 rx=0x7fda50002f70 tx=0x7fda50008de0).stop 2020-05-18T21:40:29.861 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.859+0000 7fda5ffff700 1 -- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda60140590 msgr2=0x7fda6009b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.861 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.859+0000 7fda5ffff700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda60140590 0x7fda6009b660 secure :-1 s=READY pgs=79 cs=0 l=1 rx=0x7fda600a3850 tx=0x7fda5800c850).stop 2020-05-18T21:40:29.861 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.859+0000 7fda5ffff700 1 -- 172.21.15.196:0/590803725 shutdown_connections 2020-05-18T21:40:29.862 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.859+0000 7fda5ffff700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda60140590 0x7fda6009b660 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.862 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.859+0000 7fda5ffff700 1 --2- 172.21.15.196:0/590803725 >> v2:172.21.15.80:3300/0 conn(0x7fda600a7220 0x7fda600a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.862 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.859+0000 7fda5ffff700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fda4404f9e0 0x7fda44051ea0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.862 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.859+0000 7fda5ffff700 1 --2- 172.21.15.196:0/590803725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda600a3dd0 0x7fda6013db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.862 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.859+0000 7fda5ffff700 1 -- 172.21.15.196:0/590803725 >> 172.21.15.196:0/590803725 conn(0x7fda6005a7c0 msgr2=0x7fda600a8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:29.863 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.859+0000 7fda5ffff700 1 -- 172.21.15.196:0/590803725 shutdown_connections 2020-05-18T21:40:29.863 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.859+0000 7fda5ffff700 1 -- 172.21.15.196:0/590803725 wait complete. 2020-05-18T21:40:29.869 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-05-18T21:40:29.870 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-05-18T21:40:29.970 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89cc998700 1 Processor -- start 2020-05-18T21:40:29.970 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89cc998700 1 -- start start 2020-05-18T21:40:29.973 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89cc998700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f89c80a6e50 0x7f89c80ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:29.973 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89cc998700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c81400c0 0x7f89c8140550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:29.980 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89cc998700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c80aa050 con 0x7f89c80aa6f0 2020-05-18T21:40:29.981 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89cc998700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c80a6590 con 0x7f89c80a6e50 2020-05-18T21:40:29.981 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89cc998700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80ab6b0 con 0x7f89c81400c0 2020-05-18T21:40:29.981 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c5d9b700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f89c80a6e50 0x7f89c80ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.981 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c5d9b700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f89c80a6e50 0x7f89c80ab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34220/0 (socket says 172.21.15.196:34220) 2020-05-18T21:40:29.981 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c5d9b700 1 -- 172.21.15.196:0/3270807312 learned_addr learned my addr 172.21.15.196:0/3270807312 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:29.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c6d9d700 1 --2- 172.21.15.196:0/3270807312 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c81400c0 0x7f89c8140550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c5d9b700 1 -- 172.21.15.196:0/3270807312 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c81400c0 msgr2=0x7f89c8140550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:29.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c5d9b700 1 --2- 172.21.15.196:0/3270807312 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c81400c0 0x7f89c8140550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:29.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c5d9b700 1 -- 172.21.15.196:0/3270807312 >> v1:172.21.15.80:6789/0 conn(0x7f89c80aa6f0 legacy=0x7f89c80a9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c5d9b700 1 -- 172.21.15.196:0/3270807312 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c80ab920 con 0x7f89c80a6e50 2020-05-18T21:40:29.982 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c5d9b700 1 --2- 172.21.15.196:0/3270807312 >> v2:172.21.15.196:3300/0 conn(0x7f89c80a6e50 0x7f89c80ab250 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f89c80aab60 tx=0x7f89bc008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c5d9b700 1 --2- 172.21.15.196:0/3270807312 >> v2:172.21.15.196:3300/0 conn(0x7f89c80a6e50 0x7f89c80ab250 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f89c80aab60 tx=0x7f89bc008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c5d9b700 1 --2- 172.21.15.196:0/3270807312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89c80a6e50 0x7f89c80ab250 secure :-1 s=READY pgs=80 cs=0 l=1 rx=0x7f89c80aab60 tx=0x7f89bc008920).ready entity=mon.1 client_cookie=bce8444ef61afacc server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:29.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c5d9b700 1 --2- 172.21.15.196:0/3270807312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89c80a6e50 0x7f89c80ab250 secure :-1 s=THROTTLE_DONE pgs=80 cs=0 l=1 rx=0x7f89c80aab60 tx=0x7f89bc008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c5d9b700 1 --2- 172.21.15.196:0/3270807312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89c80a6e50 0x7f89c80ab250 secure :-1 s=THROTTLE_DONE pgs=80 cs=0 l=1 rx=0x7f89c80aab60 tx=0x7f89bc008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.983 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c559a700 1 -- 172.21.15.196:0/3270807312 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89bc008e50 con 0x7f89c80a6e50 2020-05-18T21:40:29.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c5d9b700 1 --2- 172.21.15.196:0/3270807312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89c80a6e50 0x7f89c80ab250 secure :-1 s=THROTTLE_DONE pgs=80 cs=0 l=1 rx=0x7f89c80aab60 tx=0x7f89bc008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c559a700 1 -- 172.21.15.196:0/3270807312 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc010750 con 0x7f89c80a6e50 2020-05-18T21:40:29.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89c559a700 1 -- 172.21.15.196:0/3270807312 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89bc010b70 con 0x7f89c80a6e50 2020-05-18T21:40:29.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89cc998700 1 -- 172.21.15.196:0/3270807312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89c80a6e50 msgr2=0x7f89c80ab250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.984 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.971+0000 7f89cc998700 1 --2- 172.21.15.196:0/3270807312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89c80a6e50 0x7f89c80ab250 secure :-1 s=READY pgs=80 cs=0 l=1 rx=0x7f89c80aab60 tx=0x7f89bc008920).stop 2020-05-18T21:40:29.989 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89cc998700 1 -- 172.21.15.196:0/3270807312 shutdown_connections 2020-05-18T21:40:29.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89cc998700 1 --2- 172.21.15.196:0/3270807312 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c81400c0 0x7f89c8140550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:29.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89cc998700 1 --2- 172.21.15.196:0/3270807312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89c80a6e50 0x7f89c80ab250 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89cc998700 1 -- 172.21.15.196:0/3270807312 >> 172.21.15.196:0/3270807312 conn(0x7f89c8069ac0 msgr2=0x7f89c8069ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:29.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89cc998700 1 -- 172.21.15.196:0/3270807312 shutdown_connections 2020-05-18T21:40:29.990 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89cc998700 1 -- 172.21.15.196:0/3270807312 wait complete. 2020-05-18T21:40:29.991 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89cc998700 1 Processor -- start 2020-05-18T21:40:29.991 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89cc998700 1 -- start start 2020-05-18T21:40:29.991 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89cc998700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c80a6e50 0x7f89c809fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.991 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89cc998700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c8062210 con 0x7f89c80aa6f0 2020-05-18T21:40:29.991 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89cc998700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c809ff00 con 0x7f89c81400c0 2020-05-18T21:40:29.992 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89cc998700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c8062470 con 0x7f89c80a6e50 2020-05-18T21:40:29.992 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89c6d9d700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f89c81400c0 0x7f89c813f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:42988/0 (socket says 172.21.15.196:42988) 2020-05-18T21:40:29.992 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89c6d9d700 1 -- 172.21.15.196:0/3338938936 learned_addr learned my addr 172.21.15.196:0/3338938936 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:29.992 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c80a6e50 0x7f89c809fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.992 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166781465 0 0) 0x7f89c809ff00 con 0x7f89c81400c0 2020-05-18T21:40:29.993 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89a4002fe0 con 0x7f89c81400c0 2020-05-18T21:40:29.993 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200014449 0 0) 0x7f89c8062210 con 0x7f89c80aa6f0 2020-05-18T21:40:29.993 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89c809ff00 con 0x7f89c80aa6f0 2020-05-18T21:40:29.993 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3922240352 0 0) 0x7f89a4002fe0 con 0x7f89c81400c0 2020-05-18T21:40:29.993 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 --> v1:172.21.15.196:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f89c8062210 con 0x7f89c81400c0 2020-05-18T21:40:29.993 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7f89c0001850 con 0x7f89c81400c0 2020-05-18T21:40:29.994 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c80a6e50 msgr2=0x7f89c809fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.994 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c80a6e50 0x7f89c809fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.994 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 >> v1:172.21.15.196:6789/0 conn(0x7f89c81400c0 legacy=0x7f89c813f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.994 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 >> v1:172.21.15.80:6789/0 conn(0x7f89c80aa6f0 legacy=0x7f89c80a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.994 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 0x7f89a4004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.995 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89a4003b30 0x7f89a4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.995 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c80a6e50 0x7f89a4004410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:29.995 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 0x7f89a4004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.995 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c8062470 con 0x7f89a4003b30 2020-05-18T21:40:29.995 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40048a0 con 0x7f89a4004bf0 2020-05-18T21:40:29.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.975+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a400b760 con 0x7f89c80a6e50 2020-05-18T21:40:29.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c5d9b700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89a4003b30 0x7f89a4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c6d9d700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c80a6e50 0x7f89a4004410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:29.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c659c700 1 -- 172.21.15.196:0/3338938936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c80a6e50 msgr2=0x7f89a4004410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c80a6e50 0x7f89a4004410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c659c700 1 -- 172.21.15.196:0/3338938936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89a4003b30 msgr2=0x7f89a4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:29.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89a4003b30 0x7f89a4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:29.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c659c700 1 -- 172.21.15.196:0/3338938936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c8062710 con 0x7f89a4004bf0 2020-05-18T21:40:29.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89cc998700 1 -- 172.21.15.196:0/3338938936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c80629b0 con 0x7f89a4004bf0 2020-05-18T21:40:29.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c6d9d700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c80a6e50 0x7f89a4004410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-05-18T21:40:29.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89cc998700 1 -- 172.21.15.196:0/3338938936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c8066020 con 0x7f89a4004bf0 2020-05-18T21:40:29.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 0x7f89a4004fd0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f89c80a6860 tx=0x7f89b8010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 0x7f89a4004fd0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f89c80a6860 tx=0x7f89b8010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 0x7f89a4004fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rx=0x7f89c80a6860 tx=0x7f89b8010c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:29.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 0x7f89a4004fd0 secure :-1 s=THROTTLE_DONE pgs=82 cs=0 l=1 rx=0x7f89c80a6860 tx=0x7f89b8010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89b8006710 con 0x7f89a4004bf0 2020-05-18T21:40:29.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 0x7f89a4004fd0 secure :-1 s=THROTTLE_DONE pgs=82 cs=0 l=1 rx=0x7f89c80a6860 tx=0x7f89b8010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b8006870 con 0x7f89a4004bf0 2020-05-18T21:40:29.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 0x7f89a4004fd0 secure :-1 s=THROTTLE_DONE pgs=82 cs=0 l=1 rx=0x7f89c80a6860 tx=0x7f89b8010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:29.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89b8006c50 con 0x7f89a4004bf0 2020-05-18T21:40:29.999 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89cc998700 1 -- 172.21.15.196:0/3338938936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a0003380 con 0x7f89a4004bf0 2020-05-18T21:40:30.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 0x7f89a4004fd0 secure :-1 s=THROTTLE_DONE pgs=82 cs=0 l=1 rx=0x7f89c80a6860 tx=0x7f89b8010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:30.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f89b8012340 con 0x7f89a4004bf0 2020-05-18T21:40:30.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 0x7f89a4004fd0 secure :-1 s=THROTTLE_DONE pgs=82 cs=0 l=1 rx=0x7f89c80a6860 tx=0x7f89b8010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:30.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.979+0000 7f89b77fe700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f89a4057f10 0x7f89a405a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:30.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.983+0000 7f89c5d9b700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f89a4057f10 0x7f89a405a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:30.000 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.983+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f89b8063af0 con 0x7f89a4004bf0 2020-05-18T21:40:30.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.983+0000 7f89c5d9b700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f89a4057f10 0x7f89a405a3d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f89bc008500 tx=0x7f89c80a2800).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:30.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.983+0000 7f89c5d9b700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f89a4057f10 0x7f89a405a3d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f89bc008500 tx=0x7f89c80a2800).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:30.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.983+0000 7f89c5d9b700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f89a4057f10 0x7f89a405a3d0 secure :-1 s=READY pgs=32 cs=0 l=1 rx=0x7f89bc008500 tx=0x7f89c80a2800).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:30.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.987+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 0x7f89a4004fd0 secure :-1 s=THROTTLE_DONE pgs=82 cs=0 l=1 rx=0x7f89c80a6860 tx=0x7f89b8010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:30.001 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:29.987+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f89b803de20 con 0x7f89a4004bf0 2020-05-18T21:40:30.102 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:30.103+0000 7f89cc998700 1 -- 172.21.15.196:0/3338938936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f89a0002240 con 0x7f89a4004bf0 2020-05-18T21:40:30.899 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:30.899+0000 7f89c659c700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 0x7f89a4004fd0 secure :-1 s=THROTTLE_DONE pgs=82 cs=0 l=1 rx=0x7f89c80a6860 tx=0x7f89b8010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:30.899 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:30.899+0000 7f89b77fe700 1 -- 172.21.15.196:0/3338938936 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v19) v1 ==== 133+0+0 (secure 0 0 0) 0x7f89b8041440 con 0x7f89a4004bf0 2020-05-18T21:40:30.902 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:30.903+0000 7f89cc998700 1 -- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f89a4057f10 msgr2=0x7f89a405a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:30.903 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:30.903+0000 7f89cc998700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f89a4057f10 0x7f89a405a3d0 secure :-1 s=READY pgs=32 cs=0 l=1 rx=0x7f89bc008500 tx=0x7f89c80a2800).stop 2020-05-18T21:40:30.903 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:30.903+0000 7f89cc998700 1 -- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 msgr2=0x7f89a4004fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:30.903 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:30.903+0000 7f89cc998700 1 --2- 172.21.15.196:0/3338938936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89a4004bf0 0x7f89a4004fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rx=0x7f89c80a6860 tx=0x7f89b8010c30).stop 2020-05-18T21:40:30.903 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:30.903+0000 7f89c6d9d700 1 -- 172.21.15.196:0/3338938936 reap_dead start 2020-05-18T21:40:30.905 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:30.903+0000 7f89cc998700 1 -- 172.21.15.196:0/3338938936 shutdown_connections 2020-05-18T21:40:30.905 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:30.903+0000 7f89cc998700 1 -- 172.21.15.196:0/3338938936 >> 172.21.15.196:0/3338938936 conn(0x7f89c8069ac0 msgr2=0x7f89c805eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:30.905 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:30.907+0000 7f89cc998700 1 -- 172.21.15.196:0/3338938936 shutdown_connections 2020-05-18T21:40:30.906 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:30.907+0000 7f89cc998700 1 -- 172.21.15.196:0/3338938936 wait complete. 2020-05-18T21:40:30.907 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'cephfs_metadata' created 2020-05-18T21:40:30.918 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-05-18T21:40:31.021 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.019+0000 7f03c0de6700 1 Processor -- start 2020-05-18T21:40:31.021 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.019+0000 7f03c0de6700 1 -- start start 2020-05-18T21:40:31.024 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03c0de6700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03bc06a510 0x7f03bc0a81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:31.024 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03c0de6700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f03bc134f80 0x7f03bc135410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:31.024 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03c0de6700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc1392a0 con 0x7f03bc138e40 2020-05-18T21:40:31.024 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03c0de6700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03bc0a6770 con 0x7f03bc134f80 2020-05-18T21:40:31.025 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03c0de6700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc067200 con 0x7f03bc06a510 2020-05-18T21:40:31.025 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03bad9d700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f03bc134f80 0x7f03bc135410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:31.025 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03bad9d700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f03bc134f80 0x7f03bc135410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34242/0 (socket says 172.21.15.196:34242) 2020-05-18T21:40:31.025 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03bad9d700 1 -- 172.21.15.196:0/451225888 learned_addr learned my addr 172.21.15.196:0/451225888 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:31.025 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03ba59c700 1 --2- 172.21.15.196:0/451225888 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03bc06a510 0x7f03bc0a81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:31.026 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03bad9d700 1 -- 172.21.15.196:0/451225888 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03bc06a510 msgr2=0x7f03bc0a81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:31.026 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03bad9d700 1 --2- 172.21.15.196:0/451225888 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03bc06a510 0x7f03bc0a81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:31.026 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03bad9d700 1 -- 172.21.15.196:0/451225888 >> v1:172.21.15.80:6789/0 conn(0x7f03bc138e40 legacy=0x7f03bc0a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:31.026 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03bad9d700 1 -- 172.21.15.196:0/451225888 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc138700 con 0x7f03bc134f80 2020-05-18T21:40:31.026 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03bad9d700 1 --2- 172.21.15.196:0/451225888 >> v2:172.21.15.196:3300/0 conn(0x7f03bc134f80 0x7f03bc135410 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f03b40032f0 tx=0x7f03b4007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.026 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03bad9d700 1 --2- 172.21.15.196:0/451225888 >> v2:172.21.15.196:3300/0 conn(0x7f03bc134f80 0x7f03bc135410 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f03b40032f0 tx=0x7f03b4007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.027 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03bad9d700 1 --2- 172.21.15.196:0/451225888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc135410 secure :-1 s=READY pgs=83 cs=0 l=1 rx=0x7f03b40032f0 tx=0x7f03b4007e00).ready entity=mon.1 client_cookie=ee4c798a04508676 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:31.027 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03bad9d700 1 --2- 172.21.15.196:0/451225888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc135410 secure :-1 s=THROTTLE_DONE pgs=83 cs=0 l=1 rx=0x7f03b40032f0 tx=0x7f03b4007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.027 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03bad9d700 1 --2- 172.21.15.196:0/451225888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc135410 secure :-1 s=THROTTLE_DONE pgs=83 cs=0 l=1 rx=0x7f03b40032f0 tx=0x7f03b4007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.027 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03b9d9b700 1 -- 172.21.15.196:0/451225888 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03b4011dc0 con 0x7f03bc134f80 2020-05-18T21:40:31.027 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03bad9d700 1 --2- 172.21.15.196:0/451225888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc135410 secure :-1 s=THROTTLE_DONE pgs=83 cs=0 l=1 rx=0x7f03b40032f0 tx=0x7f03b4007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.028 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03b9d9b700 1 -- 172.21.15.196:0/451225888 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b400a180 con 0x7f03bc134f80 2020-05-18T21:40:31.028 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03b9d9b700 1 -- 172.21.15.196:0/451225888 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03b400a5a0 con 0x7f03bc134f80 2020-05-18T21:40:31.029 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03c0de6700 1 -- 172.21.15.196:0/451225888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 msgr2=0x7f03bc135410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:31.029 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03c0de6700 1 --2- 172.21.15.196:0/451225888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc135410 secure :-1 s=READY pgs=83 cs=0 l=1 rx=0x7f03b40032f0 tx=0x7f03b4007e00).stop 2020-05-18T21:40:31.031 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03c0de6700 1 -- 172.21.15.196:0/451225888 shutdown_connections 2020-05-18T21:40:31.032 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03c0de6700 1 --2- 172.21.15.196:0/451225888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc135410 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:31.032 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03c0de6700 1 --2- 172.21.15.196:0/451225888 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03bc06a510 0x7f03bc0a81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:31.032 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03c0de6700 1 -- 172.21.15.196:0/451225888 >> 172.21.15.196:0/451225888 conn(0x7f03bc05fba0 msgr2=0x7f03bc069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:31.032 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03c0de6700 1 -- 172.21.15.196:0/451225888 shutdown_connections 2020-05-18T21:40:31.033 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.023+0000 7f03c0de6700 1 -- 172.21.15.196:0/451225888 wait complete. 2020-05-18T21:40:31.033 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03c0de6700 1 Processor -- start 2020-05-18T21:40:31.033 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03c0de6700 1 -- start start 2020-05-18T21:40:31.033 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03c0de6700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f03bc134f80 0x7f03bc1499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:31.033 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03c0de6700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc0a1720 con 0x7f03bc06a510 2020-05-18T21:40:31.034 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03c0de6700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc0a19c0 con 0x7f03bc138e40 2020-05-18T21:40:31.034 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03c0de6700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03bc0a1510 con 0x7f03bc134f80 2020-05-18T21:40:31.034 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b3fff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f03bc134f80 0x7f03bc1499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:31.034 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03bad9d700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f03bc138e40 0x7f03bc1312c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43008/0 (socket says 172.21.15.196:43008) 2020-05-18T21:40:31.034 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03bad9d700 1 -- 172.21.15.196:0/1946705675 learned_addr learned my addr 172.21.15.196:0/1946705675 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:31.034 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b3fff700 1 -- 172.21.15.196:0/1946705675 >> v1:172.21.15.196:6789/0 conn(0x7f03bc138e40 legacy=0x7f03bc1312c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:31.035 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b3fff700 1 -- 172.21.15.196:0/1946705675 >> v1:172.21.15.80:6789/0 conn(0x7f03bc06a510 legacy=0x7f03bc1492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:31.035 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b37fe700 1 -- 172.21.15.196:0/1946705675 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829431115 0 0) 0x7f03bc0a19c0 con 0x7f03bc138e40 2020-05-18T21:40:31.035 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b3fff700 1 -- 172.21.15.196:0/1946705675 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc09f6e0 con 0x7f03bc134f80 2020-05-18T21:40:31.035 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b3fff700 1 --2- 172.21.15.196:0/1946705675 >> v2:172.21.15.196:3300/0 conn(0x7f03bc134f80 0x7f03bc1499b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f03bc066db0 tx=0x7f03a40109d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.035 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b3fff700 1 --2- 172.21.15.196:0/1946705675 >> v2:172.21.15.196:3300/0 conn(0x7f03bc134f80 0x7f03bc1499b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f03bc066db0 tx=0x7f03a40109d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.036 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b3fff700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc1499b0 secure :-1 s=READY pgs=85 cs=0 l=1 rx=0x7f03bc066db0 tx=0x7f03a40109d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:31.036 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b3fff700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc1499b0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f03bc066db0 tx=0x7f03a40109d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.036 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b3fff700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc1499b0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f03bc066db0 tx=0x7f03a40109d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.036 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b3fff700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc1499b0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f03bc066db0 tx=0x7f03a40109d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.036 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b37fe700 1 -- 172.21.15.196:0/1946705675 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03a400cd30 con 0x7f03bc134f80 2020-05-18T21:40:31.037 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b37fe700 1 -- 172.21.15.196:0/1946705675 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a400ce90 con 0x7f03bc134f80 2020-05-18T21:40:31.037 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03c0de6700 1 -- 172.21.15.196:0/1946705675 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc09f980 con 0x7f03bc134f80 2020-05-18T21:40:31.037 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03b37fe700 1 -- 172.21.15.196:0/1946705675 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03a4012260 con 0x7f03bc134f80 2020-05-18T21:40:31.037 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.027+0000 7f03c0de6700 1 -- 172.21.15.196:0/1946705675 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc0a0a20 con 0x7f03bc134f80 2020-05-18T21:40:31.037 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.031+0000 7f03b3fff700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc1499b0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f03bc066db0 tx=0x7f03a40109d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.037 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.031+0000 7f03c0de6700 1 -- 172.21.15.196:0/1946705675 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03bc0569e0 con 0x7f03bc134f80 2020-05-18T21:40:31.038 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.031+0000 7f03b3fff700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc1499b0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f03bc066db0 tx=0x7f03a40109d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.038 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.031+0000 7f03b37fe700 1 -- 172.21.15.196:0/1946705675 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f03a4006520 con 0x7f03bc134f80 2020-05-18T21:40:31.038 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.031+0000 7f03b37fe700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03a004f970 0x7f03a0051e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:31.038 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.031+0000 7f03b37fe700 1 -- 172.21.15.196:0/1946705675 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f03a40638c0 con 0x7f03bc134f80 2020-05-18T21:40:31.039 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.031+0000 7f03ba59c700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03a004f970 0x7f03a0051e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:31.040 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.031+0000 7f03ba59c700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03a004f970 0x7f03a0051e30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f03bc138100 tx=0x7f03ac008d50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.040 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.031+0000 7f03ba59c700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03a004f970 0x7f03a0051e30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f03bc138100 tx=0x7f03ac008d50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.041 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.031+0000 7f03ba59c700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03a004f970 0x7f03a0051e30 secure :-1 s=READY pgs=33 cs=0 l=1 rx=0x7f03bc138100 tx=0x7f03ac008d50).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:31.041 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.035+0000 7f03b3fff700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc1499b0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f03bc066db0 tx=0x7f03a40109d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.041 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.035+0000 7f03b37fe700 1 -- 172.21.15.196:0/1946705675 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f03a403dbb0 con 0x7f03bc134f80 2020-05-18T21:40:31.150 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.151+0000 7f03c0de6700 1 -- 172.21.15.196:0/1946705675 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f03bc066fe0 con 0x7f03bc134f80 2020-05-18T21:40:31.908 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.907+0000 7f03b3fff700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc1499b0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f03bc066db0 tx=0x7f03a40109d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:31.909 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.911+0000 7f03b37fe700 1 -- 172.21.15.196:0/1946705675 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v20) v1 ==== 125+0+0 (secure 0 0 0) 0x7f03a40411d0 con 0x7f03bc134f80 2020-05-18T21:40:31.918 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.919+0000 7f03c0de6700 1 -- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03a004f970 msgr2=0x7f03a0051e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:31.918 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.919+0000 7f03c0de6700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03a004f970 0x7f03a0051e30 secure :-1 s=READY pgs=33 cs=0 l=1 rx=0x7f03bc138100 tx=0x7f03ac008d50).stop 2020-05-18T21:40:31.918 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.919+0000 7f03c0de6700 1 -- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 msgr2=0x7f03bc1499b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:31.919 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.919+0000 7f03c0de6700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc1499b0 secure :-1 s=READY pgs=85 cs=0 l=1 rx=0x7f03bc066db0 tx=0x7f03a40109d0).stop 2020-05-18T21:40:31.919 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.919+0000 7f03c0de6700 1 -- 172.21.15.196:0/1946705675 shutdown_connections 2020-05-18T21:40:31.919 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.919+0000 7f03c0de6700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03a004f970 0x7f03a0051e30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:31.919 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.919+0000 7f03c0de6700 1 --2- 172.21.15.196:0/1946705675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03bc134f80 0x7f03bc1499b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:31.919 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.919+0000 7f03c0de6700 1 -- 172.21.15.196:0/1946705675 >> 172.21.15.196:0/1946705675 conn(0x7f03bc05fba0 msgr2=0x7f03bc133900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:31.920 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.919+0000 7f03c0de6700 1 -- 172.21.15.196:0/1946705675 shutdown_connections 2020-05-18T21:40:31.920 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:31.919+0000 7f03c0de6700 1 -- 172.21.15.196:0/1946705675 wait complete. 2020-05-18T21:40:31.920 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'cephfs_data' created 2020-05-18T21:40:31.928 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-05-18T21:40:32.035 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d50873700 1 Processor -- start 2020-05-18T21:40:32.035 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d50873700 1 -- start start 2020-05-18T21:40:32.038 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d50873700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3d4c13fd80 0x7f3d4c140210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:32.039 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d50873700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3d4c140670 0x7f3d4c1471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:32.039 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d50873700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4c0a7750 con 0x7f3d4c05b210 2020-05-18T21:40:32.039 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d50873700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d4c0a4e10 con 0x7f3d4c13fd80 2020-05-18T21:40:32.039 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d50873700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d4c0a7980 con 0x7f3d4c140670 2020-05-18T21:40:32.039 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4bfff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3d4c140670 0x7f3d4c1471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.040 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4bfff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3d4c140670 0x7f3d4c1471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34256/0 (socket says 172.21.15.196:34256) 2020-05-18T21:40:32.040 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4bfff700 1 -- 172.21.15.196:0/1921516069 learned_addr learned my addr 172.21.15.196:0/1921516069 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:32.040 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4affd700 1 --2- 172.21.15.196:0/1921516069 >> v2:172.21.15.80:3300/0 conn(0x7f3d4c13fd80 0x7f3d4c140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.040 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4bfff700 1 -- 172.21.15.196:0/1921516069 >> v2:172.21.15.80:3300/0 conn(0x7f3d4c13fd80 msgr2=0x7f3d4c140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:32.040 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4bfff700 1 --2- 172.21.15.196:0/1921516069 >> v2:172.21.15.80:3300/0 conn(0x7f3d4c13fd80 0x7f3d4c140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:32.041 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4bfff700 1 -- 172.21.15.196:0/1921516069 >> v1:172.21.15.80:6789/0 conn(0x7f3d4c05b210 legacy=0x7f3d4c0a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.041 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4bfff700 1 -- 172.21.15.196:0/1921516069 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c0a7c20 con 0x7f3d4c140670 2020-05-18T21:40:32.041 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4bfff700 1 --2- 172.21.15.196:0/1921516069 >> v2:172.21.15.196:3300/0 conn(0x7f3d4c140670 0x7f3d4c1471e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f3d44002f70 tx=0x7f3d44007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.041 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4bfff700 1 --2- 172.21.15.196:0/1921516069 >> v2:172.21.15.196:3300/0 conn(0x7f3d4c140670 0x7f3d4c1471e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f3d44002f70 tx=0x7f3d44007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.042 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4bfff700 1 --2- 172.21.15.196:0/1921516069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c140670 0x7f3d4c1471e0 secure :-1 s=READY pgs=86 cs=0 l=1 rx=0x7f3d44002f70 tx=0x7f3d44007a80).ready entity=mon.1 client_cookie=9becf04bd176c574 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:32.042 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4bfff700 1 --2- 172.21.15.196:0/1921516069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c140670 0x7f3d4c1471e0 secure :-1 s=THROTTLE_DONE pgs=86 cs=0 l=1 rx=0x7f3d44002f70 tx=0x7f3d44007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.042 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4bfff700 1 --2- 172.21.15.196:0/1921516069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c140670 0x7f3d4c1471e0 secure :-1 s=THROTTLE_DONE pgs=86 cs=0 l=1 rx=0x7f3d44002f70 tx=0x7f3d44007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.043 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4bfff700 1 --2- 172.21.15.196:0/1921516069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c140670 0x7f3d4c1471e0 secure :-1 s=THROTTLE_DONE pgs=86 cs=0 l=1 rx=0x7f3d44002f70 tx=0x7f3d44007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.043 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4a7fc700 1 -- 172.21.15.196:0/1921516069 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d44011760 con 0x7f3d4c140670 2020-05-18T21:40:32.043 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4a7fc700 1 -- 172.21.15.196:0/1921516069 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d440118c0 con 0x7f3d4c140670 2020-05-18T21:40:32.043 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d50873700 1 -- 172.21.15.196:0/1921516069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c140670 msgr2=0x7f3d4c1471e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.043 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d4a7fc700 1 -- 172.21.15.196:0/1921516069 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d44011ce0 con 0x7f3d4c140670 2020-05-18T21:40:32.044 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.035+0000 7f3d50873700 1 --2- 172.21.15.196:0/1921516069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c140670 0x7f3d4c1471e0 secure :-1 s=READY pgs=86 cs=0 l=1 rx=0x7f3d44002f70 tx=0x7f3d44007a80).stop 2020-05-18T21:40:32.044 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.039+0000 7f3d50873700 1 -- 172.21.15.196:0/1921516069 shutdown_connections 2020-05-18T21:40:32.044 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.039+0000 7f3d50873700 1 --2- 172.21.15.196:0/1921516069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c140670 0x7f3d4c1471e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:32.044 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.039+0000 7f3d50873700 1 --2- 172.21.15.196:0/1921516069 >> v2:172.21.15.80:3300/0 conn(0x7f3d4c13fd80 0x7f3d4c140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:32.044 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.039+0000 7f3d50873700 1 -- 172.21.15.196:0/1921516069 >> 172.21.15.196:0/1921516069 conn(0x7f3d4c060640 msgr2=0x7f3d4c05a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:32.045 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.039+0000 7f3d50873700 1 -- 172.21.15.196:0/1921516069 shutdown_connections 2020-05-18T21:40:32.045 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.039+0000 7f3d50873700 1 -- 172.21.15.196:0/1921516069 wait complete. 2020-05-18T21:40:32.048 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.039+0000 7f3d50873700 1 Processor -- start 2020-05-18T21:40:32.048 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.039+0000 7f3d50873700 1 -- start start 2020-05-18T21:40:32.048 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d50873700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:32.049 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d50873700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3d4c140670 0x7f3d4c13a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:32.049 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d50873700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4c0a6190 con 0x7f3d4c05b210 2020-05-18T21:40:32.049 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d50873700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d4c064370 con 0x7f3d4c140670 2020-05-18T21:40:32.049 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d50873700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c0633e0 con 0x7f3d4c13fd80 2020-05-18T21:40:32.050 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4b7fe700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f3d4c05b210 0x7f3d4c061010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.196:38846/0 (socket says 172.21.15.196:38846) 2020-05-18T21:40:32.050 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4affd700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.050 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4b7fe700 1 -- 172.21.15.196:0/2136425126 learned_addr learned my addr 172.21.15.196:0/2136425126 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:32.050 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4bfff700 1 --2- 172.21.15.196:0/2136425126 >> v2:172.21.15.80:3300/0 conn(0x7f3d4c140670 0x7f3d4c13a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.050 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4affd700 1 -- 172.21.15.196:0/2136425126 >> v2:172.21.15.80:3300/0 conn(0x7f3d4c140670 msgr2=0x7f3d4c13a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.051 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4affd700 1 --2- 172.21.15.196:0/2136425126 >> v2:172.21.15.80:3300/0 conn(0x7f3d4c140670 0x7f3d4c13a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:32.051 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4affd700 1 -- 172.21.15.196:0/2136425126 >> v1:172.21.15.80:6789/0 conn(0x7f3d4c05b210 legacy=0x7f3d4c061010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.051 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4affd700 1 -- 172.21.15.196:0/2136425126 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c063680 con 0x7f3d4c13fd80 2020-05-18T21:40:32.051 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4affd700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f3d3c0032f0 tx=0x7f3d3c010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.051 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4affd700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f3d3c0032f0 tx=0x7f3d3c010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.052 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4affd700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 secure :-1 s=READY pgs=21 cs=0 l=1 rx=0x7f3d3c0032f0 tx=0x7f3d3c010f70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:32.052 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4affd700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 secure :-1 s=THROTTLE_DONE pgs=21 cs=0 l=1 rx=0x7f3d3c0032f0 tx=0x7f3d3c010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.052 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d48ff9700 1 -- 172.21.15.196:0/2136425126 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d3c005e70 con 0x7f3d4c13fd80 2020-05-18T21:40:32.052 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4affd700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 secure :-1 s=THROTTLE_DONE pgs=21 cs=0 l=1 rx=0x7f3d3c0032f0 tx=0x7f3d3c010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.052 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d48ff9700 1 -- 172.21.15.196:0/2136425126 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c007560 con 0x7f3d4c13fd80 2020-05-18T21:40:32.053 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4affd700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 secure :-1 s=THROTTLE_DONE pgs=21 cs=0 l=1 rx=0x7f3d3c0032f0 tx=0x7f3d3c010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.053 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d50873700 1 -- 172.21.15.196:0/2136425126 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4c061da0 con 0x7f3d4c13fd80 2020-05-18T21:40:32.053 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d48ff9700 1 -- 172.21.15.196:0/2136425126 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d3c007900 con 0x7f3d4c13fd80 2020-05-18T21:40:32.053 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d50873700 1 -- 172.21.15.196:0/2136425126 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4c063c30 con 0x7f3d4c13fd80 2020-05-18T21:40:32.054 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d50873700 1 -- 172.21.15.196:0/2136425126 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4c0569e0 con 0x7f3d4c13fd80 2020-05-18T21:40:32.054 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4affd700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 secure :-1 s=THROTTLE_DONE pgs=21 cs=0 l=1 rx=0x7f3d3c0032f0 tx=0x7f3d3c010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.054 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d48ff9700 1 -- 172.21.15.196:0/2136425126 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f3d3c0101a0 con 0x7f3d4c13fd80 2020-05-18T21:40:32.054 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.043+0000 7f3d4affd700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 secure :-1 s=THROTTLE_DONE pgs=21 cs=0 l=1 rx=0x7f3d3c0032f0 tx=0x7f3d3c010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.054 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.047+0000 7f3d48ff9700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f3d3004f9c0 0x7f3d30051e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:32.054 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.047+0000 7f3d48ff9700 1 -- 172.21.15.196:0/2136425126 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f3d3c0090b0 con 0x7f3d4c13fd80 2020-05-18T21:40:32.055 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.047+0000 7f3d4b7fe700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f3d3004f9c0 0x7f3d30051e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.055 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.047+0000 7f3d4b7fe700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f3d3004f9c0 0x7f3d30051e80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f3d4c062bc0 tx=0x7f3d34001650).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.055 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.047+0000 7f3d4b7fe700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f3d3004f9c0 0x7f3d30051e80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f3d4c062bc0 tx=0x7f3d34001650).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.055 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.047+0000 7f3d4b7fe700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f3d3004f9c0 0x7f3d30051e80 secure :-1 s=READY pgs=34 cs=0 l=1 rx=0x7f3d4c062bc0 tx=0x7f3d34001650).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:32.056 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.055+0000 7f3d4affd700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 secure :-1 s=THROTTLE_DONE pgs=21 cs=0 l=1 rx=0x7f3d3c0032f0 tx=0x7f3d3c010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.056 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.055+0000 7f3d48ff9700 1 -- 172.21.15.196:0/2136425126 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f3d3c03e410 con 0x7f3d4c13fd80 2020-05-18T21:40:32.204 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.203+0000 7f3d50873700 1 -- 172.21.15.196:0/2136425126 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f3d4c0a53a0 con 0x7f3d4c13fd80 2020-05-18T21:40:32.207 INFO:tasks.ceph.mon.b.smithi080.stderr:2020-05-18T21:40:32.205+0000 7fef9ee46700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-05-18T21:40:32.217 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.219+0000 7f3d4affd700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 secure :-1 s=THROTTLE_DONE pgs=21 cs=0 l=1 rx=0x7f3d3c0032f0 tx=0x7f3d3c010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.218 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.219+0000 7f3d48ff9700 1 -- 172.21.15.196:0/2136425126 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v2) v1 ==== 187+0+0 (secure 0 0 0) 0x7f3d3c041a30 con 0x7f3d4c13fd80 2020-05-18T21:40:32.223 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.223+0000 7f3d50873700 1 -- 172.21.15.196:0/2136425126 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f3d3004f9c0 msgr2=0x7f3d30051e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.223 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.223+0000 7f3d50873700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f3d3004f9c0 0x7f3d30051e80 secure :-1 s=READY pgs=34 cs=0 l=1 rx=0x7f3d4c062bc0 tx=0x7f3d34001650).stop 2020-05-18T21:40:32.223 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.223+0000 7f3d50873700 1 -- 172.21.15.196:0/2136425126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 msgr2=0x7f3d4c0a8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.223 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.223+0000 7f3d50873700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 secure :-1 s=READY pgs=21 cs=0 l=1 rx=0x7f3d3c0032f0 tx=0x7f3d3c010f70).stop 2020-05-18T21:40:32.224 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.223+0000 7f3d50873700 1 -- 172.21.15.196:0/2136425126 shutdown_connections 2020-05-18T21:40:32.224 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.223+0000 7f3d50873700 1 --2- 172.21.15.196:0/2136425126 >> v2:172.21.15.80:3300/0 conn(0x7f3d4c140670 0x7f3d4c13a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:32.224 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.223+0000 7f3d50873700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f3d3004f9c0 0x7f3d30051e80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:32.224 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.223+0000 7f3d50873700 1 --2- 172.21.15.196:0/2136425126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c13fd80 0x7f3d4c0a8b30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:32.224 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.223+0000 7f3d50873700 1 -- 172.21.15.196:0/2136425126 >> 172.21.15.196:0/2136425126 conn(0x7f3d4c060640 msgr2=0x7f3d4c05eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:32.225 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.223+0000 7f3d50873700 1 -- 172.21.15.196:0/2136425126 shutdown_connections 2020-05-18T21:40:32.225 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.223+0000 7f3d50873700 1 -- 172.21.15.196:0/2136425126 wait complete. 2020-05-18T21:40:32.225 INFO:teuthology.orchestra.run.smithi196.stderr:new fs with metadata pool 3 and data pool 4 2020-05-18T21:40:32.234 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-05-18T21:40:32.327 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facbd4a9700 1 Processor -- start 2020-05-18T21:40:32.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facbd4a9700 1 -- start start 2020-05-18T21:40:32.331 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facbd4a9700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb8066910 0x7facb80aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:32.331 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facbd4a9700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb80a9360 con 0x7facb813a750 2020-05-18T21:40:32.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facbd4a9700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb80aab80 con 0x7facb8065f70 2020-05-18T21:40:32.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facbd4a9700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb80a9d60 con 0x7facb8066910 2020-05-18T21:40:32.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facb7fff700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7facb8065f70 0x7facb805a9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43028/0 (socket says 172.21.15.196:43028) 2020-05-18T21:40:32.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facb7fff700 1 -- 172.21.15.196:0/3661041663 learned_addr learned my addr 172.21.15.196:0/3661041663 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:32.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facb77fe700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb8066910 0x7facb80aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810544560 0 0) 0x7facb80aab80 con 0x7facb8065f70 2020-05-18T21:40:32.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faca4003270 con 0x7facb8065f70 2020-05-18T21:40:32.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 985527672 0 0) 0x7faca4003270 con 0x7facb8065f70 2020-05-18T21:40:32.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb8066910 msgr2=0x7facb80aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:32.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facb6ffd700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb8066910 0x7facb80aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:32.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 >> v1:172.21.15.80:6789/0 conn(0x7facb813a750 legacy=0x7facb813ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb80a8a60 con 0x7facb8065f70 2020-05-18T21:40:32.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7facac001710 con 0x7facb8065f70 2020-05-18T21:40:32.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 >> v1:172.21.15.196:6789/0 conn(0x7facb8065f70 legacy=0x7facb805a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.327+0000 7facb6ffd700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca4003760 0x7faca4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:32.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb6ffd700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faca4004010 0x7faca4004510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:32.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb6ffd700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb8066910 0x7faca4004ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:32.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb80a9d60 con 0x7faca4004010 2020-05-18T21:40:32.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4004f90 con 0x7faca4003760 2020-05-18T21:40:32.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca4005150 con 0x7facb8066910 2020-05-18T21:40:32.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb7fff700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca4003760 0x7faca4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facbcca8700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb8066910 0x7faca4004ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb77fe700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faca4004010 0x7faca4004510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb7fff700 1 -- 172.21.15.196:0/3661041663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb8066910 msgr2=0x7faca4004ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:32.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb7fff700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb8066910 0x7faca4004ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:32.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb7fff700 1 -- 172.21.15.196:0/3661041663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faca4004010 msgr2=0x7faca4004510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:32.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb7fff700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faca4004010 0x7faca4004510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:32.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb7fff700 1 -- 172.21.15.196:0/3661041663 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca4003270 con 0x7faca4003760 2020-05-18T21:40:32.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb7fff700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca4003760 0x7faca4003b40 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7facac001420 tx=0x7facac001d20).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb7fff700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca4003760 0x7faca4003b40 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7facac001420 tx=0x7facac001d20).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb7fff700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca4003760 0x7faca4003b40 secure :-1 s=READY pgs=88 cs=0 l=1 rx=0x7facac001420 tx=0x7facac001d20).ready entity=mon.1 client_cookie=d418697e4d0da41d server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:32.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb7fff700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca4003760 0x7faca4003b40 secure :-1 s=THROTTLE_DONE pgs=88 cs=0 l=1 rx=0x7facac001420 tx=0x7facac001d20).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb7fff700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca4003760 0x7faca4003b40 secure :-1 s=THROTTLE_DONE pgs=88 cs=0 l=1 rx=0x7facac001420 tx=0x7facac001d20).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facac00ce20 con 0x7faca4003760 2020-05-18T21:40:32.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facac00e370 con 0x7faca4003760 2020-05-18T21:40:32.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb7fff700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca4003760 0x7faca4003b40 secure :-1 s=THROTTLE_DONE pgs=88 cs=0 l=1 rx=0x7facac001420 tx=0x7facac001d20).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.339 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facbd4a9700 1 -- 172.21.15.196:0/3661041663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca4003760 msgr2=0x7faca4003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.339 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facb6ffd700 1 -- 172.21.15.196:0/3661041663 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facac0100c0 con 0x7faca4003760 2020-05-18T21:40:32.339 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facbd4a9700 1 --2- 172.21.15.196:0/3661041663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faca4003760 0x7faca4003b40 secure :-1 s=READY pgs=88 cs=0 l=1 rx=0x7facac001420 tx=0x7facac001d20).stop 2020-05-18T21:40:32.339 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facbcca8700 1 -- 172.21.15.196:0/3661041663 reap_dead start 2020-05-18T21:40:32.340 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facbd4a9700 1 -- 172.21.15.196:0/3661041663 shutdown_connections 2020-05-18T21:40:32.340 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facbd4a9700 1 -- 172.21.15.196:0/3661041663 >> 172.21.15.196:0/3661041663 conn(0x7facb80ae900 msgr2=0x7facb80aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:32.340 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facbd4a9700 1 -- 172.21.15.196:0/3661041663 shutdown_connections 2020-05-18T21:40:32.340 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facbd4a9700 1 -- 172.21.15.196:0/3661041663 wait complete. 2020-05-18T21:40:32.344 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facbd4a9700 1 Processor -- start 2020-05-18T21:40:32.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.331+0000 7facbd4a9700 1 -- start start 2020-05-18T21:40:32.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbd4a9700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb80a3620 0x7facb81494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:32.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbd4a9700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb80ae4d0 con 0x7facb80a0f70 2020-05-18T21:40:32.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbd4a9700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb813a560 con 0x7facb80a08c0 2020-05-18T21:40:32.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbd4a9700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb8148ae0 con 0x7facb80a3620 2020-05-18T21:40:32.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb7fff700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7facb80a08c0 0x7facb8139740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43038/0 (socket says 172.21.15.196:43038) 2020-05-18T21:40:32.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb7fff700 1 -- 172.21.15.196:0/2305837581 learned_addr learned my addr 172.21.15.196:0/2305837581 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:32.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb80a3620 0x7facb81494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.347 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379823956 0 0) 0x7facb813a560 con 0x7facb80a08c0 2020-05-18T21:40:32.347 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac98002fe0 con 0x7facb80a08c0 2020-05-18T21:40:32.347 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2715709555 0 0) 0x7fac98002fe0 con 0x7facb80a08c0 2020-05-18T21:40:32.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 --> v1:172.21.15.196:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7facb813a560 con 0x7facb80a08c0 2020-05-18T21:40:32.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7facac000ed0 con 0x7facb80a08c0 2020-05-18T21:40:32.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb80a3620 msgr2=0x7facb81494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb80a3620 0x7facb81494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:32.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 >> v1:172.21.15.196:6789/0 conn(0x7facb80a08c0 legacy=0x7facb8139740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 >> v1:172.21.15.80:6789/0 conn(0x7facb80a0f70 legacy=0x7facb8139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.349 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb80a3620 0x7fac98004a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:32.349 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac98004ee0 0x7fac980039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:32.349 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 0x7fac98004280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:32.349 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb8148ae0 con 0x7fac98004ee0 2020-05-18T21:40:32.349 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac98004830 con 0x7fac98003e00 2020-05-18T21:40:32.350 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac98005410 con 0x7facb80a3620 2020-05-18T21:40:32.350 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 0x7fac98004280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.350 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb77fe700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac98004ee0 0x7fac980039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.350 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb7fff700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb80a3620 0x7fac98004a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.350 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 -- 172.21.15.196:0/2305837581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb80a3620 msgr2=0x7fac98004a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facb80a3620 0x7fac98004a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:32.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 -- 172.21.15.196:0/2305837581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac98004ee0 msgr2=0x7fac980039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac98004ee0 0x7fac980039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:32.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 -- 172.21.15.196:0/2305837581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb8148d80 con 0x7fac98003e00 2020-05-18T21:40:32.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbd4a9700 1 -- 172.21.15.196:0/2305837581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb8149020 con 0x7fac98003e00 2020-05-18T21:40:32.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 0x7fac98004280 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7facac001110 tx=0x7faca8010ce0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbd4a9700 1 -- 172.21.15.196:0/2305837581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb80a1930 con 0x7fac98003e00 2020-05-18T21:40:32.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 0x7fac98004280 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7facac001110 tx=0x7faca8010ce0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.353 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 0x7fac98004280 secure :-1 s=READY pgs=90 cs=0 l=1 rx=0x7facac001110 tx=0x7faca8010ce0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:32.353 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 0x7fac98004280 secure :-1 s=THROTTLE_DONE pgs=90 cs=0 l=1 rx=0x7facac001110 tx=0x7faca8010ce0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.353 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faca8002940 con 0x7fac98003e00 2020-05-18T21:40:32.353 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 0x7fac98004280 secure :-1 s=THROTTLE_DONE pgs=90 cs=0 l=1 rx=0x7facac001110 tx=0x7faca8010ce0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.353 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 0x7fac98004280 secure :-1 s=THROTTLE_DONE pgs=90 cs=0 l=1 rx=0x7facac001110 tx=0x7faca8010ce0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.354 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca8002aa0 con 0x7fac98003e00 2020-05-18T21:40:32.354 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faca8006520 con 0x7fac98003e00 2020-05-18T21:40:32.354 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 0x7fac98004280 secure :-1 s=THROTTLE_DONE pgs=90 cs=0 l=1 rx=0x7facac001110 tx=0x7faca8010ce0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.354 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbd4a9700 1 -- 172.21.15.196:0/2305837581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac94003380 con 0x7fac98003e00 2020-05-18T21:40:32.354 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7faca8006680 con 0x7fac98003e00 2020-05-18T21:40:32.354 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.335+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 0x7fac98004280 secure :-1 s=THROTTLE_DONE pgs=90 cs=0 l=1 rx=0x7facac001110 tx=0x7faca8010ce0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.339+0000 7facb57fa700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fac98057b80 0x7fac9805a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:32.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.339+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faca8063b10 con 0x7fac98003e00 2020-05-18T21:40:32.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.339+0000 7facb7fff700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fac98057b80 0x7fac9805a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.339+0000 7facb7fff700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fac98057b80 0x7fac9805a040 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7facac00ba00 tx=0x7facac001d20).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.339+0000 7facb7fff700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fac98057b80 0x7fac9805a040 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7facac00ba00 tx=0x7facac001d20).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.356 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.339+0000 7facb7fff700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fac98057b80 0x7fac9805a040 secure :-1 s=READY pgs=35 cs=0 l=1 rx=0x7facac00ba00 tx=0x7facac001d20).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:32.356 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.343+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 0x7fac98004280 secure :-1 s=THROTTLE_DONE pgs=90 cs=0 l=1 rx=0x7facac001110 tx=0x7faca8010ce0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.356 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.343+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7faca803dfc0 con 0x7fac98003e00 2020-05-18T21:40:32.456 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.455+0000 7facbd4a9700 1 -- 172.21.15.196:0/2305837581 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac94002210 con 0x7fac98003e00 2020-05-18T21:40:32.458 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.459+0000 7facbcca8700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 0x7fac98004280 secure :-1 s=THROTTLE_DONE pgs=90 cs=0 l=1 rx=0x7facac001110 tx=0x7faca8010ce0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.458 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.459+0000 7facb57fa700 1 -- 172.21.15.196:0/2305837581 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16102 (secure 0 0 0) 0x7faca80415e0 con 0x7fac98003e00 2020-05-18T21:40:32.459 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:32.459 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":21,"fsid":"d048673a-3d91-4cc6-882d-e9cc70e7cd6c","created":"2020-05-18T21:40:15.911738+0000","modified":"2020-05-18T21:40:32.207737+0000","last_up_change":"2020-05-18T21:40:22.762927+0000","last_in_change":"2020-05-18T21:40:22.762927+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-05-18T21:40:19.808941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-05-18T21:40:26.944311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-05-18T21:40:30.135661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-05-18T21:40:31.183488+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8db63a0a-61dc-42b5-9559-fddcf436cab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":12771},{"type":"v1","addr":"172.21.15.196:6801","nonce":12771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":12771},{"type":"v1","addr":"172.21.15.196:6803","nonce":12771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":12771},{"type":"v1","addr":"172.21.15.196:6807","nonce":12771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":12771},{"type":"v1","addr":"172.21.15.196:6805","nonce":12771}]},"public_addr":"172.21.15.196:6801/12771","cluster_addr":"172.21.15.196:6803/12771","heartbeat_back_addr":"172.21.15.196:6807/12771","heartbeat_front_addr":"172.21.15.196:6805/12771","state":["exists","up"]},{"osd":1,"uuid":"6773b0c3-9f49-42a2-b75b-c10c734ff086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":12772},{"type":"v1","addr":"172.21.15.196:6817","nonce":12772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":12772},{"type":"v1","addr":"172.21.15.196:6819","nonce":12772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":12772},{"type":"v1","addr":"172.21.15.196:6823","nonce":12772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":12772},{"type":"v1","addr":"172.21.15.196:6821","nonce":12772}]},"public_addr":"172.21.15.196:6817/12772","cluster_addr":"172.21.15.196:6819/12772","heartbeat_back_addr":"172.21.15.196:6823/12772","heartbeat_front_addr":"172.21.15.196:6821/12772","state":["exists","up"]},{"osd":2,"uuid":"ac1f2e05-9bb7-4768-b712-10e4d2ea4154","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":12773},{"type":"v1","addr":"172.21.15.196:6809","nonce":12773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":12773},{"type":"v1","addr":"172.21.15.196:6811","nonce":12773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":12773},{"type":"v1","addr":"172.21.15.196:6815","nonce":12773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":12773},{"type":"v1","addr":"172.21.15.196:6813","nonce":12773}]},"public_addr":"172.21.15.196:6809/12773","cluster_addr":"172.21.15.196:6811/12773","heartbeat_back_addr":"172.21.15.196:6815/12773","heartbeat_front_addr":"172.21.15.196:6813/12773","state":["exists","up"]},{"osd":3,"uuid":"bae5ab40-235b-4e0f-8d0d-524afd099b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":12775},{"type":"v1","addr":"172.21.15.196:6825","nonce":12775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":12775},{"type":"v1","addr":"172.21.15.196:6827","nonce":12775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":12775},{"type":"v1","addr":"172.21.15.196:6831","nonce":12775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":12775},{"type":"v1","addr":"172.21.15.196:6829","nonce":12775}]},"public_addr":"172.21.15.196:6825/12775","cluster_addr":"172.21.15.196:6827/12775","heartbeat_back_addr":"172.21.15.196:6831/12775","heartbeat_front_addr":"172.21.15.196:6829/12775","state":["exists","up"]},{"osd":4,"uuid":"275fc52c-58e4-4f35-a0cb-e117fe36bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":13040},{"type":"v1","addr":"172.21.15.80:6819","nonce":13040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6821","nonce":13040},{"type":"v1","addr":"172.21.15.80:6823","nonce":13040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6829","nonce":13040},{"type":"v1","addr":"172.21.15.80:6831","nonce":13040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":13040},{"type":"v1","addr":"172.21.15.80:6827","nonce":13040}]},"public_addr":"172.21.15.80:6819/13040","cluster_addr":"172.21.15.80:6823/13040","heartbeat_back_addr":"172.21.15.80:6831/13040","heartbeat_front_addr":"172.21.15.80:6827/13040","state":["exists","up"]},{"osd":5,"uuid":"9795b89c-e2f8-4abf-b9c7-796727a57002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":13037},{"type":"v1","addr":"172.21.15.80:6818","nonce":13037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":13037},{"type":"v1","addr":"172.21.15.80:6822","nonce":13037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":13037},{"type":"v1","addr":"172.21.15.80:6830","nonce":13037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":13037},{"type":"v1","addr":"172.21.15.80:6826","nonce":13037}]},"public_addr":"172.21.15.80:6818/13037","cluster_addr":"172.21.15.80:6822/13037","heartbeat_back_addr":"172.21.15.80:6830/13037","heartbeat_front_addr":"172.21.15.80:6826/13037","state":["exists","up"]},{"osd":6,"uuid":"e42df2e8-9cc5-4c7e-a243-bd18ccfbd002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":13039},{"type":"v1","addr":"172.21.15.80:6804","nonce":13039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":13039},{"type":"v1","addr":"172.21.15.80:6808","nonce":13039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":13039},{"type":"v1","addr":"172.21.15.80:6813","nonce":13039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":13039},{"type":"v1","addr":"172.21.15.80:6810","nonce":13039}]},"public_addr":"172.21.15.80:6804/13039","cluster_addr":"172.21.15.80:6808/13039","heartbeat_back_addr":"172.21.15.80:6813/13039","heartbeat_front_addr":"172.21.15.80:6810/13039","state":["exists","up"]},{"osd":7,"uuid":"ebd6c938-beff-4668-b70e-076ed22bb8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":13038},{"type":"v1","addr":"172.21.15.80:6801","nonce":13038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":13038},{"type":"v1","addr":"172.21.15.80:6805","nonce":13038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":13038},{"type":"v1","addr":"172.21.15.80:6817","nonce":13038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":13038},{"type":"v1","addr":"172.21.15.80:6811","nonce":13038}]},"public_addr":"172.21.15.80:6801/13038","cluster_addr":"172.21.15.80:6805/13038","heartbeat_back_addr":"172.21.15.80:6817/13038","heartbeat_front_addr":"172.21.15.80:6811/13038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.134515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.439836+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.476825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.176642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.452301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.427183+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.230250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.210168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-18T21:40:32.460 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.459+0000 7facbd4a9700 1 -- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fac98057b80 msgr2=0x7fac9805a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.460 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.459+0000 7facbd4a9700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fac98057b80 0x7fac9805a040 secure :-1 s=READY pgs=35 cs=0 l=1 rx=0x7facac00ba00 tx=0x7facac001d20).stop 2020-05-18T21:40:32.460 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.459+0000 7facbd4a9700 1 -- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 msgr2=0x7fac98004280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.460 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.459+0000 7facbd4a9700 1 --2- 172.21.15.196:0/2305837581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac98003e00 0x7fac98004280 secure :-1 s=READY pgs=90 cs=0 l=1 rx=0x7facac001110 tx=0x7faca8010ce0).stop 2020-05-18T21:40:32.461 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.459+0000 7facbcca8700 1 -- 172.21.15.196:0/2305837581 reap_dead start 2020-05-18T21:40:32.461 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.459+0000 7facbcca8700 1 -- 172.21.15.196:0/2305837581 reap_dead start 2020-05-18T21:40:32.461 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.459+0000 7facbd4a9700 1 -- 172.21.15.196:0/2305837581 shutdown_connections 2020-05-18T21:40:32.461 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.459+0000 7facbd4a9700 1 -- 172.21.15.196:0/2305837581 >> 172.21.15.196:0/2305837581 conn(0x7facb80ae900 msgr2=0x7facb80ad280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:32.461 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.459+0000 7facbd4a9700 1 -- 172.21.15.196:0/2305837581 shutdown_connections 2020-05-18T21:40:32.462 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.459+0000 7facbd4a9700 1 -- 172.21.15.196:0/2305837581 wait complete. 2020-05-18T21:40:32.471 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-05-18T21:40:32.570 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63f65a8700 1 Processor -- start 2020-05-18T21:40:32.571 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63f65a8700 1 -- start start 2020-05-18T21:40:32.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63f65a8700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f63f00ab910 0x7f63f00a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:32.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63f65a8700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f63f0137bb0 0x7f63f01345e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:32.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63f65a8700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f0137ff0 con 0x7f63f0134a40 2020-05-18T21:40:32.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63f65a8700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f00a66d0 con 0x7f63f0137bb0 2020-05-18T21:40:32.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63f65a8700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f0138250 con 0x7f63f00ab910 2020-05-18T21:40:32.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63e7fff700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f63f0137bb0 0x7f63f01345e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63e7fff700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f63f0137bb0 0x7f63f01345e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.196:38518/0 (socket says 172.21.15.196:38518) 2020-05-18T21:40:32.576 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63e7fff700 1 -- 172.21.15.196:0/346797877 learned_addr learned my addr 172.21.15.196:0/346797877 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:32.576 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63effff700 1 --2- 172.21.15.196:0/346797877 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f63f00ab910 0x7f63f00a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.576 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63e7fff700 1 -- 172.21.15.196:0/346797877 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f63f00ab910 msgr2=0x7f63f00a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:32.577 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63e7fff700 1 --2- 172.21.15.196:0/346797877 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f63f00ab910 0x7f63f00a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:32.577 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63e7fff700 1 -- 172.21.15.196:0/346797877 >> v1:172.21.15.80:6789/0 conn(0x7f63f0134a40 legacy=0x7f63f012e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.577 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63e7fff700 1 -- 172.21.15.196:0/346797877 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f01384f0 con 0x7f63f0137bb0 2020-05-18T21:40:32.577 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63ef7fe700 1 -- 172.21.15.196:0/346797877 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678933146 0 0) 0x7f63f0137ff0 con 0x7f63f0134a40 2020-05-18T21:40:32.577 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.571+0000 7f63e7fff700 1 --2- 172.21.15.196:0/346797877 >> v2:172.21.15.80:3300/0 conn(0x7f63f0137bb0 0x7f63f01345e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f63d8002f70 tx=0x7f63d8007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.577 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63e7fff700 1 --2- 172.21.15.196:0/346797877 >> v2:172.21.15.80:3300/0 conn(0x7f63f0137bb0 0x7f63f01345e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f63d8002f70 tx=0x7f63d8007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63e7fff700 1 --2- 172.21.15.196:0/346797877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f0137bb0 0x7f63f01345e0 secure :-1 s=READY pgs=73 cs=0 l=1 rx=0x7f63d8002f70 tx=0x7f63d8007a80).ready entity=mon.0 client_cookie=2f62bb9cb88c04a4 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:32.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63e7fff700 1 --2- 172.21.15.196:0/346797877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f0137bb0 0x7f63f01345e0 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7f63d8002f70 tx=0x7f63d8007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63ef7fe700 1 -- 172.21.15.196:0/346797877 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63d80138c0 con 0x7f63f0137bb0 2020-05-18T21:40:32.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63e7fff700 1 --2- 172.21.15.196:0/346797877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f0137bb0 0x7f63f01345e0 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7f63d8002f70 tx=0x7f63d8007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63e7fff700 1 --2- 172.21.15.196:0/346797877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f0137bb0 0x7f63f01345e0 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7f63d8002f70 tx=0x7f63d8007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.579 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63ef7fe700 1 -- 172.21.15.196:0/346797877 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d8013a20 con 0x7f63f0137bb0 2020-05-18T21:40:32.579 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63ef7fe700 1 -- 172.21.15.196:0/346797877 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63d8013e00 con 0x7f63f0137bb0 2020-05-18T21:40:32.579 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 -- 172.21.15.196:0/346797877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f0137bb0 msgr2=0x7f63f01345e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.579 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 --2- 172.21.15.196:0/346797877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f0137bb0 0x7f63f01345e0 secure :-1 s=READY pgs=73 cs=0 l=1 rx=0x7f63d8002f70 tx=0x7f63d8007a80).stop 2020-05-18T21:40:32.579 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 -- 172.21.15.196:0/346797877 shutdown_connections 2020-05-18T21:40:32.580 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 --2- 172.21.15.196:0/346797877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f0137bb0 0x7f63f01345e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:32.580 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 --2- 172.21.15.196:0/346797877 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f63f00ab910 0x7f63f00a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:32.581 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 -- 172.21.15.196:0/346797877 >> 172.21.15.196:0/346797877 conn(0x7f63f0060fe0 msgr2=0x7f63f00aaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:32.581 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 -- 172.21.15.196:0/346797877 shutdown_connections 2020-05-18T21:40:32.581 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 -- 172.21.15.196:0/346797877 wait complete. 2020-05-18T21:40:32.584 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 Processor -- start 2020-05-18T21:40:32.584 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 -- start start 2020-05-18T21:40:32.584 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f63f00ab910 0x7f63f01303e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:32.584 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f009f250 con 0x7f63f0137bb0 2020-05-18T21:40:32.585 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f012eef0 con 0x7f63f0134a40 2020-05-18T21:40:32.585 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.575+0000 7f63f65a8700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f0135820 con 0x7f63f00ab910 2020-05-18T21:40:32.585 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63e7fff700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f63f0134a40 0x7f63f01410c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43058/0 (socket says 172.21.15.196:43058) 2020-05-18T21:40:32.585 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63e7fff700 1 -- 172.21.15.196:0/1737554571 learned_addr learned my addr 172.21.15.196:0/1737554571 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:32.585 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63effff700 1 --2- 172.21.15.196:0/1737554571 >> v2:172.21.15.80:3300/0 conn(0x7f63f00ab910 0x7f63f01303e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.586 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63edffb700 1 -- 172.21.15.196:0/1737554571 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632738105 0 0) 0x7f63f012eef0 con 0x7f63f0134a40 2020-05-18T21:40:32.586 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63edffb700 1 -- 172.21.15.196:0/1737554571 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63c8002fe0 con 0x7f63f0134a40 2020-05-18T21:40:32.586 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63edffb700 1 -- 172.21.15.196:0/1737554571 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503115822 0 0) 0x7f63f009f250 con 0x7f63f0137bb0 2020-05-18T21:40:32.586 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63edffb700 1 -- 172.21.15.196:0/1737554571 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63f012eef0 con 0x7f63f0137bb0 2020-05-18T21:40:32.586 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63effff700 1 -- 172.21.15.196:0/1737554571 >> v1:172.21.15.196:6789/0 conn(0x7f63f0134a40 legacy=0x7f63f01410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.587 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63effff700 1 -- 172.21.15.196:0/1737554571 >> v1:172.21.15.80:6789/0 conn(0x7f63f0137bb0 legacy=0x7f63f00a44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.587 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63effff700 1 -- 172.21.15.196:0/1737554571 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f012f190 con 0x7f63f00ab910 2020-05-18T21:40:32.587 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63effff700 1 --2- 172.21.15.196:0/1737554571 >> v2:172.21.15.80:3300/0 conn(0x7f63f00ab910 0x7f63f01303e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f63f0131c00 tx=0x7f63e0010c00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.587 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63effff700 1 --2- 172.21.15.196:0/1737554571 >> v2:172.21.15.80:3300/0 conn(0x7f63f00ab910 0x7f63f01303e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f63f0131c00 tx=0x7f63e0010c00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.587 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63effff700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f00ab910 0x7f63f01303e0 secure :-1 s=READY pgs=75 cs=0 l=1 rx=0x7f63f0131c00 tx=0x7f63e0010c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:32.587 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63effff700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f00ab910 0x7f63f01303e0 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7f63f0131c00 tx=0x7f63e0010c00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.588 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63edffb700 1 -- 172.21.15.196:0/1737554571 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63e0007550 con 0x7f63f00ab910 2020-05-18T21:40:32.588 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63effff700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f00ab910 0x7f63f01303e0 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7f63f0131c00 tx=0x7f63e0010c00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.588 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63effff700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f00ab910 0x7f63f01303e0 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7f63f0131c00 tx=0x7f63e0010c00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.588 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63edffb700 1 -- 172.21.15.196:0/1737554571 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e0005700 con 0x7f63f00ab910 2020-05-18T21:40:32.588 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63f65a8700 1 -- 172.21.15.196:0/1737554571 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f012f630 con 0x7f63f00ab910 2020-05-18T21:40:32.589 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63edffb700 1 -- 172.21.15.196:0/1737554571 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63e0005aa0 con 0x7f63f00ab910 2020-05-18T21:40:32.589 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63f65a8700 1 -- 172.21.15.196:0/1737554571 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f0141850 con 0x7f63f00ab910 2020-05-18T21:40:32.589 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63f65a8700 1 -- 172.21.15.196:0/1737554571 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f0131e40 con 0x7f63f00ab910 2020-05-18T21:40:32.589 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63effff700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f00ab910 0x7f63f01303e0 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7f63f0131c00 tx=0x7f63e0010c00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.589 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63edffb700 1 -- 172.21.15.196:0/1737554571 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f63e0005ca0 con 0x7f63f00ab910 2020-05-18T21:40:32.590 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.579+0000 7f63effff700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f00ab910 0x7f63f01303e0 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7f63f0131c00 tx=0x7f63e0010c00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.590 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.583+0000 7f63edffb700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f63c804fe10 0x7f63c80522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:32.590 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.583+0000 7f63edffb700 1 -- 172.21.15.196:0/1737554571 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f63e0063c60 con 0x7f63f00ab910 2020-05-18T21:40:32.590 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.583+0000 7f63e7fff700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f63c804fe10 0x7f63c80522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.590 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.583+0000 7f63e7fff700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f63c804fe10 0x7f63c80522d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f63d8008950 tx=0x7f63d8007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.590 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.583+0000 7f63e7fff700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f63c804fe10 0x7f63c80522d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f63d8008950 tx=0x7f63d8007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.591 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.583+0000 7f63e7fff700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f63c804fe10 0x7f63c80522d0 secure :-1 s=READY pgs=36 cs=0 l=1 rx=0x7f63d8008950 tx=0x7f63d8007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:32.591 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.587+0000 7f63effff700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f00ab910 0x7f63f01303e0 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7f63f0131c00 tx=0x7f63e0010c00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.591 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.587+0000 7f63edffb700 1 -- 172.21.15.196:0/1737554571 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f63e003df90 con 0x7f63f00ab910 2020-05-18T21:40:32.699 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.699+0000 7f63f65a8700 1 -- 172.21.15.196:0/1737554571 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63f012fd30 con 0x7f63f00ab910 2020-05-18T21:40:32.701 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.699+0000 7f63effff700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f00ab910 0x7f63f01303e0 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7f63f0131c00 tx=0x7f63e0010c00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.701 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.699+0000 7f63edffb700 1 -- 172.21.15.196:0/1737554571 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16102 (secure 0 0 0) 0x7f63e00415b0 con 0x7f63f00ab910 2020-05-18T21:40:32.702 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:32.702 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":21,"fsid":"d048673a-3d91-4cc6-882d-e9cc70e7cd6c","created":"2020-05-18T21:40:15.911738+0000","modified":"2020-05-18T21:40:32.207737+0000","last_up_change":"2020-05-18T21:40:22.762927+0000","last_in_change":"2020-05-18T21:40:22.762927+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-05-18T21:40:19.808941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-05-18T21:40:26.944311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-05-18T21:40:30.135661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-05-18T21:40:31.183488+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8db63a0a-61dc-42b5-9559-fddcf436cab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":12771},{"type":"v1","addr":"172.21.15.196:6801","nonce":12771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":12771},{"type":"v1","addr":"172.21.15.196:6803","nonce":12771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":12771},{"type":"v1","addr":"172.21.15.196:6807","nonce":12771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":12771},{"type":"v1","addr":"172.21.15.196:6805","nonce":12771}]},"public_addr":"172.21.15.196:6801/12771","cluster_addr":"172.21.15.196:6803/12771","heartbeat_back_addr":"172.21.15.196:6807/12771","heartbeat_front_addr":"172.21.15.196:6805/12771","state":["exists","up"]},{"osd":1,"uuid":"6773b0c3-9f49-42a2-b75b-c10c734ff086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":12772},{"type":"v1","addr":"172.21.15.196:6817","nonce":12772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":12772},{"type":"v1","addr":"172.21.15.196:6819","nonce":12772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":12772},{"type":"v1","addr":"172.21.15.196:6823","nonce":12772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":12772},{"type":"v1","addr":"172.21.15.196:6821","nonce":12772}]},"public_addr":"172.21.15.196:6817/12772","cluster_addr":"172.21.15.196:6819/12772","heartbeat_back_addr":"172.21.15.196:6823/12772","heartbeat_front_addr":"172.21.15.196:6821/12772","state":["exists","up"]},{"osd":2,"uuid":"ac1f2e05-9bb7-4768-b712-10e4d2ea4154","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":12773},{"type":"v1","addr":"172.21.15.196:6809","nonce":12773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":12773},{"type":"v1","addr":"172.21.15.196:6811","nonce":12773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":12773},{"type":"v1","addr":"172.21.15.196:6815","nonce":12773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":12773},{"type":"v1","addr":"172.21.15.196:6813","nonce":12773}]},"public_addr":"172.21.15.196:6809/12773","cluster_addr":"172.21.15.196:6811/12773","heartbeat_back_addr":"172.21.15.196:6815/12773","heartbeat_front_addr":"172.21.15.196:6813/12773","state":["exists","up"]},{"osd":3,"uuid":"bae5ab40-235b-4e0f-8d0d-524afd099b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":12775},{"type":"v1","addr":"172.21.15.196:6825","nonce":12775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":12775},{"type":"v1","addr":"172.21.15.196:6827","nonce":12775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":12775},{"type":"v1","addr":"172.21.15.196:6831","nonce":12775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":12775},{"type":"v1","addr":"172.21.15.196:6829","nonce":12775}]},"public_addr":"172.21.15.196:6825/12775","cluster_addr":"172.21.15.196:6827/12775","heartbeat_back_addr":"172.21.15.196:6831/12775","heartbeat_front_addr":"172.21.15.196:6829/12775","state":["exists","up"]},{"osd":4,"uuid":"275fc52c-58e4-4f35-a0cb-e117fe36bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":13040},{"type":"v1","addr":"172.21.15.80:6819","nonce":13040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6821","nonce":13040},{"type":"v1","addr":"172.21.15.80:6823","nonce":13040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6829","nonce":13040},{"type":"v1","addr":"172.21.15.80:6831","nonce":13040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":13040},{"type":"v1","addr":"172.21.15.80:6827","nonce":13040}]},"public_addr":"172.21.15.80:6819/13040","cluster_addr":"172.21.15.80:6823/13040","heartbeat_back_addr":"172.21.15.80:6831/13040","heartbeat_front_addr":"172.21.15.80:6827/13040","state":["exists","up"]},{"osd":5,"uuid":"9795b89c-e2f8-4abf-b9c7-796727a57002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":13037},{"type":"v1","addr":"172.21.15.80:6818","nonce":13037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":13037},{"type":"v1","addr":"172.21.15.80:6822","nonce":13037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":13037},{"type":"v1","addr":"172.21.15.80:6830","nonce":13037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":13037},{"type":"v1","addr":"172.21.15.80:6826","nonce":13037}]},"public_addr":"172.21.15.80:6818/13037","cluster_addr":"172.21.15.80:6822/13037","heartbeat_back_addr":"172.21.15.80:6830/13037","heartbeat_front_addr":"172.21.15.80:6826/13037","state":["exists","up"]},{"osd":6,"uuid":"e42df2e8-9cc5-4c7e-a243-bd18ccfbd002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":13039},{"type":"v1","addr":"172.21.15.80:6804","nonce":13039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":13039},{"type":"v1","addr":"172.21.15.80:6808","nonce":13039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":13039},{"type":"v1","addr":"172.21.15.80:6813","nonce":13039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":13039},{"type":"v1","addr":"172.21.15.80:6810","nonce":13039}]},"public_addr":"172.21.15.80:6804/13039","cluster_addr":"172.21.15.80:6808/13039","heartbeat_back_addr":"172.21.15.80:6813/13039","heartbeat_front_addr":"172.21.15.80:6810/13039","state":["exists","up"]},{"osd":7,"uuid":"ebd6c938-beff-4668-b70e-076ed22bb8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":13038},{"type":"v1","addr":"172.21.15.80:6801","nonce":13038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":13038},{"type":"v1","addr":"172.21.15.80:6805","nonce":13038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":13038},{"type":"v1","addr":"172.21.15.80:6817","nonce":13038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":13038},{"type":"v1","addr":"172.21.15.80:6811","nonce":13038}]},"public_addr":"172.21.15.80:6801/13038","cluster_addr":"172.21.15.80:6805/13038","heartbeat_back_addr":"172.21.15.80:6817/13038","heartbeat_front_addr":"172.21.15.80:6811/13038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.134515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.439836+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.476825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.176642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.452301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.427183+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.230250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.210168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-18T21:40:32.703 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.703+0000 7f63f65a8700 1 -- 172.21.15.196:0/1737554571 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f63c804fe10 msgr2=0x7f63c80522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.703 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.703+0000 7f63f65a8700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f63c804fe10 0x7f63c80522d0 secure :-1 s=READY pgs=36 cs=0 l=1 rx=0x7f63d8008950 tx=0x7f63d8007a80).stop 2020-05-18T21:40:32.703 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.703+0000 7f63f65a8700 1 -- 172.21.15.196:0/1737554571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f00ab910 msgr2=0x7f63f01303e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.704 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.703+0000 7f63f65a8700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f00ab910 0x7f63f01303e0 secure :-1 s=READY pgs=75 cs=0 l=1 rx=0x7f63f0131c00 tx=0x7f63e0010c00).stop 2020-05-18T21:40:32.704 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.703+0000 7f63f65a8700 1 -- 172.21.15.196:0/1737554571 shutdown_connections 2020-05-18T21:40:32.704 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.703+0000 7f63f65a8700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f63c804fe10 0x7f63c80522d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:32.704 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.703+0000 7f63f65a8700 1 --2- 172.21.15.196:0/1737554571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63f00ab910 0x7f63f01303e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:32.704 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.703+0000 7f63f65a8700 1 -- 172.21.15.196:0/1737554571 >> 172.21.15.196:0/1737554571 conn(0x7f63f0060fe0 msgr2=0x7f63f0069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:32.705 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.703+0000 7f63f65a8700 1 -- 172.21.15.196:0/1737554571 shutdown_connections 2020-05-18T21:40:32.705 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.703+0000 7f63f65a8700 1 -- 172.21.15.196:0/1737554571 wait complete. 2020-05-18T21:40:32.714 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-05-18T21:40:32.821 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.819+0000 7f754ffff700 1 Processor -- start 2020-05-18T21:40:32.821 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.819+0000 7f754ffff700 1 -- start start 2020-05-18T21:40:32.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754ffff700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f75500a2d80 0x7f75501366f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:32.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754ffff700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7550136b50 con 0x7f75500aae90 2020-05-18T21:40:32.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754ffff700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75500a6680 con 0x7f75500a58f0 2020-05-18T21:40:32.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754ffff700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75500a6920 con 0x7f75500a2d80 2020-05-18T21:40:32.829 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754e7fc700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f75500a58f0 0x7f7550135fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43068/0 (socket says 172.21.15.196:43068) 2020-05-18T21:40:32.829 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754e7fc700 1 -- 172.21.15.196:0/3097927167 learned_addr learned my addr 172.21.15.196:0/3097927167 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:32.829 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754f7fe700 1 --2- 172.21.15.196:0/3097927167 >> v2:172.21.15.80:3300/0 conn(0x7f75500a2d80 0x7f75501366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.829 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900181175 0 0) 0x7f75500a6680 con 0x7f75500a58f0 2020-05-18T21:40:32.830 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7534003270 con 0x7f75500a58f0 2020-05-18T21:40:32.830 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058225324 0 0) 0x7f7550136b50 con 0x7f75500aae90 2020-05-18T21:40:32.830 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75500a6680 con 0x7f75500aae90 2020-05-18T21:40:32.830 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3925085773 0 0) 0x7f7534003270 con 0x7f75500a58f0 2020-05-18T21:40:32.830 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 >> v2:172.21.15.80:3300/0 conn(0x7f75500a2d80 msgr2=0x7f75501366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:32.831 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 --2- 172.21.15.196:0/3097927167 >> v2:172.21.15.80:3300/0 conn(0x7f75500a2d80 0x7f75501366f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:32.831 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 >> v1:172.21.15.80:6789/0 conn(0x7f75500aae90 legacy=0x7f75500a51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.831 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75500a6a60 con 0x7f75500a58f0 2020-05-18T21:40:32.831 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7f7540001a90 con 0x7f75500a58f0 2020-05-18T21:40:32.831 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 >> v1:172.21.15.196:6789/0 conn(0x7f75500a58f0 legacy=0x7f7550135fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.832 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340038f0 0x7f7534003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:32.832 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7534004510 0x7f7534004990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:32.832 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534004df0 0x7f753400b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:32.832 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75500a6920 con 0x7f7534004df0 2020-05-18T21:40:32.832 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75340053e0 con 0x7f75340038f0 2020-05-18T21:40:32.832 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753400bc40 con 0x7f7534004510 2020-05-18T21:40:32.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754f7fe700 1 --2- 172.21.15.196:0/3097927167 >> v2:172.21.15.80:3300/0 conn(0x7f75500a2d80 0x7f75501366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:32.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754effd700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340038f0 0x7f7534003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754e7fc700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7534004510 0x7f7534004990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754effd700 1 -- 172.21.15.196:0/3097927167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7534004510 msgr2=0x7f7534004990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:32.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754effd700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7534004510 0x7f7534004990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:32.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754f7fe700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534004df0 0x7f753400b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754effd700 1 -- 172.21.15.196:0/3097927167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534004df0 msgr2=0x7f753400b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:32.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754effd700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534004df0 0x7f753400b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:32.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754f7fe700 1 -- 172.21.15.196:0/3097927167 reap_dead start 2020-05-18T21:40:32.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754effd700 1 -- 172.21.15.196:0/3097927167 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7534003270 con 0x7f75340038f0 2020-05-18T21:40:32.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754effd700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340038f0 0x7f7534003cf0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f7544001110 tx=0x7f75440015d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754effd700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340038f0 0x7f7534003cf0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f7544001110 tx=0x7f75440015d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754effd700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340038f0 0x7f7534003cf0 secure :-1 s=READY pgs=93 cs=0 l=1 rx=0x7f7544001110 tx=0x7f75440015d0).ready entity=mon.1 client_cookie=ab4f40b49307ce96 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:32.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754effd700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340038f0 0x7f7534003cf0 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f7544001110 tx=0x7f75440015d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754effd700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340038f0 0x7f7534003cf0 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f7544001110 tx=0x7f75440015d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7544010d20 con 0x7f75340038f0 2020-05-18T21:40:32.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754effd700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340038f0 0x7f7534003cf0 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f7544001110 tx=0x7f75440015d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754400c560 con 0x7f75340038f0 2020-05-18T21:40:32.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754dffb700 1 -- 172.21.15.196:0/3097927167 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f754400c940 con 0x7f75340038f0 2020-05-18T21:40:32.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754ffff700 1 -- 172.21.15.196:0/3097927167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340038f0 msgr2=0x7f7534003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754ffff700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340038f0 0x7f7534003cf0 secure :-1 s=READY pgs=93 cs=0 l=1 rx=0x7f7544001110 tx=0x7f75440015d0).stop 2020-05-18T21:40:32.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754ffff700 1 -- 172.21.15.196:0/3097927167 shutdown_connections 2020-05-18T21:40:32.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754ffff700 1 --2- 172.21.15.196:0/3097927167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75340038f0 0x7f7534003cf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:32.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754ffff700 1 -- 172.21.15.196:0/3097927167 >> 172.21.15.196:0/3097927167 conn(0x7f755009d850 msgr2=0x7f75500aa440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:32.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754ffff700 1 -- 172.21.15.196:0/3097927167 shutdown_connections 2020-05-18T21:40:32.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.823+0000 7f754ffff700 1 -- 172.21.15.196:0/3097927167 wait complete. 2020-05-18T21:40:32.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754ffff700 1 Processor -- start 2020-05-18T21:40:32.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754ffff700 1 -- start start 2020-05-18T21:40:32.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754ffff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7550138780 0x7f7550061ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:32.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754ffff700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f755014a0a0 0x7f7550066660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:32.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754ffff700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f755005c020 con 0x7f75340038f0 2020-05-18T21:40:32.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754ffff700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75500a0320 con 0x7f7550138780 2020-05-18T21:40:32.840 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754ffff700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75500a05c0 con 0x7f755014a0a0 2020-05-18T21:40:32.840 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754effd700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f75340038f0 0x7f7550149990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43078/0 (socket says 172.21.15.196:43078) 2020-05-18T21:40:32.840 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754effd700 1 -- 172.21.15.196:0/1292201663 learned_addr learned my addr 172.21.15.196:0/1292201663 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:32.840 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754e7fc700 1 --2- 172.21.15.196:0/1292201663 >> v2:172.21.15.196:3300/0 conn(0x7f7550138780 0x7f7550061ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754f7fe700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f755014a0a0 0x7f7550066660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f753ffff700 1 -- 172.21.15.196:0/1292201663 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161430155 0 0) 0x7f755005c020 con 0x7f75340038f0 2020-05-18T21:40:32.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f753ffff700 1 -- 172.21.15.196:0/1292201663 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f752c002fe0 con 0x7f75340038f0 2020-05-18T21:40:32.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754e7fc700 1 -- 172.21.15.196:0/1292201663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f755014a0a0 msgr2=0x7f7550066660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754e7fc700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f755014a0a0 0x7f7550066660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:32.842 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754e7fc700 1 -- 172.21.15.196:0/1292201663 >> v1:172.21.15.196:6789/0 conn(0x7f75340038f0 legacy=0x7f7550149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:32.842 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754e7fc700 1 -- 172.21.15.196:0/1292201663 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75500a00f0 con 0x7f7550138780 2020-05-18T21:40:32.842 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754e7fc700 1 --2- 172.21.15.196:0/1292201663 >> v2:172.21.15.196:3300/0 conn(0x7f7550138780 0x7f7550061ed0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7540001860 tx=0x7f75500a6bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.842 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754e7fc700 1 --2- 172.21.15.196:0/1292201663 >> v2:172.21.15.196:3300/0 conn(0x7f7550138780 0x7f7550061ed0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7540001860 tx=0x7f75500a6bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.842 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754e7fc700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7550138780 0x7f7550061ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rx=0x7f7540001860 tx=0x7f75500a6bf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:32.843 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754e7fc700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7550138780 0x7f7550061ed0 secure :-1 s=THROTTLE_DONE pgs=95 cs=0 l=1 rx=0x7f7540001860 tx=0x7f75500a6bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.843 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f753ffff700 1 -- 172.21.15.196:0/1292201663 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f754000ce20 con 0x7f7550138780 2020-05-18T21:40:32.843 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754e7fc700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7550138780 0x7f7550061ed0 secure :-1 s=THROTTLE_DONE pgs=95 cs=0 l=1 rx=0x7f7540001860 tx=0x7f75500a6bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.843 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754e7fc700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7550138780 0x7f7550061ed0 secure :-1 s=THROTTLE_DONE pgs=95 cs=0 l=1 rx=0x7f7540001860 tx=0x7f75500a6bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.843 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f753ffff700 1 -- 172.21.15.196:0/1292201663 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7540004aa0 con 0x7f7550138780 2020-05-18T21:40:32.844 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754ffff700 1 -- 172.21.15.196:0/1292201663 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75501391b0 con 0x7f7550138780 2020-05-18T21:40:32.844 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f753ffff700 1 -- 172.21.15.196:0/1292201663 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7540004e00 con 0x7f7550138780 2020-05-18T21:40:32.844 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754ffff700 1 -- 172.21.15.196:0/1292201663 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7550066c60 con 0x7f7550138780 2020-05-18T21:40:32.844 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754e7fc700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7550138780 0x7f7550061ed0 secure :-1 s=THROTTLE_DONE pgs=95 cs=0 l=1 rx=0x7f7540001860 tx=0x7f75500a6bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.845 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754ffff700 1 -- 172.21.15.196:0/1292201663 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75500569e0 con 0x7f7550138780 2020-05-18T21:40:32.846 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f753ffff700 1 -- 172.21.15.196:0/1292201663 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f75400061a0 con 0x7f7550138780 2020-05-18T21:40:32.846 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.827+0000 7f754e7fc700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7550138780 0x7f7550061ed0 secure :-1 s=THROTTLE_DONE pgs=95 cs=0 l=1 rx=0x7f7540001860 tx=0x7f75500a6bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.846 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.831+0000 7f753ffff700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f752c04fd90 0x7f752c052250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:32.846 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.831+0000 7f753ffff700 1 -- 172.21.15.196:0/1292201663 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7540063a70 con 0x7f7550138780 2020-05-18T21:40:32.846 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.831+0000 7f754effd700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f752c04fd90 0x7f752c052250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:32.847 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.831+0000 7f754effd700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f752c04fd90 0x7f752c052250 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f75440012c0 tx=0x7f75440015d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.847 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.831+0000 7f754effd700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f752c04fd90 0x7f752c052250 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f75440012c0 tx=0x7f75440015d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.847 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.831+0000 7f754effd700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f752c04fd90 0x7f752c052250 secure :-1 s=READY pgs=37 cs=0 l=1 rx=0x7f75440012c0 tx=0x7f75440015d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:32.847 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.835+0000 7f754e7fc700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7550138780 0x7f7550061ed0 secure :-1 s=THROTTLE_DONE pgs=95 cs=0 l=1 rx=0x7f7540001860 tx=0x7f75500a6bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:32.847 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.835+0000 7f753ffff700 1 -- 172.21.15.196:0/1292201663 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f754003de00 con 0x7f7550138780 2020-05-18T21:40:32.980 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:32.979+0000 7f754ffff700 1 -- 172.21.15.196:0/1292201663 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f75500a0730 con 0x7f7550138780 2020-05-18T21:40:33.826 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.831+0000 7f754e7fc700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7550138780 0x7f7550061ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rx=0x7f7540001860 tx=0x7f75500a6bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:33.928 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.927+0000 7f754e7fc700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7550138780 0x7f7550061ed0 secure :-1 s=THROTTLE_DONE pgs=95 cs=0 l=1 rx=0x7f7540001860 tx=0x7f75500a6bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:33.928 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.927+0000 7f753ffff700 1 -- 172.21.15.196:0/1292201663 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v3) v1 ==== 118+0+0 (secure 0 0 0) 0x7f75500a0730 con 0x7f7550138780 2020-05-18T21:40:33.931 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.931+0000 7f754ffff700 1 -- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f752c04fd90 msgr2=0x7f752c052250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:33.931 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.931+0000 7f754ffff700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f752c04fd90 0x7f752c052250 secure :-1 s=READY pgs=37 cs=0 l=1 rx=0x7f75440012c0 tx=0x7f75440015d0).stop 2020-05-18T21:40:33.932 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.931+0000 7f754ffff700 1 -- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7550138780 msgr2=0x7f7550061ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:33.932 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.931+0000 7f754ffff700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7550138780 0x7f7550061ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rx=0x7f7540001860 tx=0x7f75500a6bf0).stop 2020-05-18T21:40:33.932 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.931+0000 7f754ffff700 1 -- 172.21.15.196:0/1292201663 shutdown_connections 2020-05-18T21:40:33.932 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.931+0000 7f754ffff700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f752c04fd90 0x7f752c052250 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:33.932 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.931+0000 7f754ffff700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f755014a0a0 0x7f7550066660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:33.933 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.931+0000 7f754ffff700 1 --2- 172.21.15.196:0/1292201663 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7550138780 0x7f7550061ed0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:33.933 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.931+0000 7f754ffff700 1 -- 172.21.15.196:0/1292201663 >> 172.21.15.196:0/1292201663 conn(0x7f755009d850 msgr2=0x7f755005eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:33.933 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.931+0000 7f754ffff700 1 -- 172.21.15.196:0/1292201663 shutdown_connections 2020-05-18T21:40:33.933 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:33.931+0000 7f754ffff700 1 -- 172.21.15.196:0/1292201663 wait complete. 2020-05-18T21:40:33.948 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-05-18T21:40:34.059 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1da5a2b700 1 Processor -- start 2020-05-18T21:40:34.060 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1da5a2b700 1 -- start start 2020-05-18T21:40:34.060 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1da5a2b700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1da0069e00 0x7f1da00a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:34.060 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1da5a2b700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1da00a6770 0x7f1da00a6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:34.060 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1da5a2b700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da00a5650 con 0x7f1da0132ea0 2020-05-18T21:40:34.060 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1da5a2b700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da01339b0 con 0x7f1da0069e00 2020-05-18T21:40:34.061 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1da5a2b700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da00a5090 con 0x7f1da00a6770 2020-05-18T21:40:34.061 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1da522a700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f1da0132ea0 0x7f1da01332a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43090/0 (socket says 172.21.15.196:43090) 2020-05-18T21:40:34.061 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1d9ffff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1da00a6770 0x7f1da00a6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.061 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1da522a700 1 -- 172.21.15.196:0/3141516614 learned_addr learned my addr 172.21.15.196:0/3141516614 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:34.061 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1da4a29700 1 --2- 172.21.15.196:0/3141516614 >> v2:172.21.15.80:3300/0 conn(0x7f1da0069e00 0x7f1da00a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.062 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1d9f7fe700 1 -- 172.21.15.196:0/3141516614 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849009905 0 0) 0x7f1d98001490 con 0x7f1da0132ea0 2020-05-18T21:40:34.062 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1d9f7fe700 1 -- 172.21.15.196:0/3141516614 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d84003270 con 0x7f1da0132ea0 2020-05-18T21:40:34.062 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1d9f7fe700 1 -- 172.21.15.196:0/3141516614 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3133386027 0 0) 0x7f1d84003270 con 0x7f1da0132ea0 2020-05-18T21:40:34.062 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1d9ffff700 1 -- 172.21.15.196:0/3141516614 >> v2:172.21.15.80:3300/0 conn(0x7f1da0069e00 msgr2=0x7f1da00a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:34.063 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1d9ffff700 1 --2- 172.21.15.196:0/3141516614 >> v2:172.21.15.80:3300/0 conn(0x7f1da0069e00 0x7f1da00a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:34.063 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1d9ffff700 1 -- 172.21.15.196:0/3141516614 >> v1:172.21.15.196:6789/0 conn(0x7f1da0132ea0 legacy=0x7f1da01332a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.063 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.055+0000 7f1d9ffff700 1 -- 172.21.15.196:0/3141516614 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da00a7040 con 0x7f1da00a6770 2020-05-18T21:40:34.063 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1d9ffff700 1 --2- 172.21.15.196:0/3141516614 >> v2:172.21.15.196:3300/0 conn(0x7f1da00a6770 0x7f1da00a6be0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f1d98001dc0 tx=0x7f1d8c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.064 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1d9ffff700 1 --2- 172.21.15.196:0/3141516614 >> v2:172.21.15.196:3300/0 conn(0x7f1da00a6770 0x7f1da00a6be0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f1d98001dc0 tx=0x7f1d8c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.064 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1d9ffff700 1 --2- 172.21.15.196:0/3141516614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1da00a6770 0x7f1da00a6be0 secure :-1 s=READY pgs=97 cs=0 l=1 rx=0x7f1d98001dc0 tx=0x7f1d8c007a80).ready entity=mon.1 client_cookie=1bc6d683473faaa8 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:34.064 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1d9ffff700 1 --2- 172.21.15.196:0/3141516614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1da00a6770 0x7f1da00a6be0 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f1d98001dc0 tx=0x7f1d8c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.064 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1d9f7fe700 1 -- 172.21.15.196:0/3141516614 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d98001c20 con 0x7f1da00a6770 2020-05-18T21:40:34.065 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1d9ffff700 1 --2- 172.21.15.196:0/3141516614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1da00a6770 0x7f1da00a6be0 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f1d98001dc0 tx=0x7f1d8c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.065 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1d9ffff700 1 --2- 172.21.15.196:0/3141516614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1da00a6770 0x7f1da00a6be0 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f1d98001dc0 tx=0x7f1d8c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.065 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1d9f7fe700 1 -- 172.21.15.196:0/3141516614 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c008b80 con 0x7f1da00a6770 2020-05-18T21:40:34.065 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1d9f7fe700 1 -- 172.21.15.196:0/3141516614 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d8c0113d0 con 0x7f1da00a6770 2020-05-18T21:40:34.065 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 -- 172.21.15.196:0/3141516614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1da00a6770 msgr2=0x7f1da00a6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.066 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 --2- 172.21.15.196:0/3141516614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1da00a6770 0x7f1da00a6be0 secure :-1 s=READY pgs=97 cs=0 l=1 rx=0x7f1d98001dc0 tx=0x7f1d8c007a80).stop 2020-05-18T21:40:34.066 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 -- 172.21.15.196:0/3141516614 shutdown_connections 2020-05-18T21:40:34.066 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 --2- 172.21.15.196:0/3141516614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1da00a6770 0x7f1da00a6be0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.066 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 --2- 172.21.15.196:0/3141516614 >> v2:172.21.15.80:3300/0 conn(0x7f1da0069e00 0x7f1da00a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:34.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 -- 172.21.15.196:0/3141516614 >> 172.21.15.196:0/3141516614 conn(0x7f1da0060e20 msgr2=0x7f1da00aacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:34.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 -- 172.21.15.196:0/3141516614 shutdown_connections 2020-05-18T21:40:34.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 -- 172.21.15.196:0/3141516614 wait complete. 2020-05-18T21:40:34.070 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 Processor -- start 2020-05-18T21:40:34.070 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 -- start start 2020-05-18T21:40:34.070 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1da0069e00 0x7f1da0065f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:34.071 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1da0132ea0 0x7f1da009eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:34.071 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da0066870 con 0x7f1da00a6770 2020-05-18T21:40:34.071 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da00a0470 con 0x7f1da0069e00 2020-05-18T21:40:34.071 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.059+0000 7f1da5a2b700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00a0710 con 0x7f1da0132ea0 2020-05-18T21:40:34.071 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1d9ffff700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1da00a6770 0x7f1da00a2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.196:38920/0 (socket says 172.21.15.196:38920) 2020-05-18T21:40:34.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1d9ffff700 1 -- 172.21.15.196:0/4227192719 learned_addr learned my addr 172.21.15.196:0/4227192719 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:34.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da522a700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1da0132ea0 0x7f1da009eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da4a29700 1 --2- 172.21.15.196:0/4227192719 >> v2:172.21.15.80:3300/0 conn(0x7f1da0069e00 0x7f1da0065f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1d9dffb700 1 -- 172.21.15.196:0/4227192719 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441039235 0 0) 0x7f1da0066870 con 0x7f1da00a6770 2020-05-18T21:40:34.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1d9dffb700 1 -- 172.21.15.196:0/4227192719 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d88002fe0 con 0x7f1da00a6770 2020-05-18T21:40:34.073 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da4a29700 1 -- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1da0132ea0 msgr2=0x7f1da009eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.073 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da4a29700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1da0132ea0 0x7f1da009eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.073 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da4a29700 1 -- 172.21.15.196:0/4227192719 >> v1:172.21.15.80:6789/0 conn(0x7f1da00a6770 legacy=0x7f1da00a2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.073 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da4a29700 1 -- 172.21.15.196:0/4227192719 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da00a0080 con 0x7f1da0069e00 2020-05-18T21:40:34.073 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da522a700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1da0132ea0 0x7f1da009eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:34.074 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da4a29700 1 --2- 172.21.15.196:0/4227192719 >> v2:172.21.15.80:3300/0 conn(0x7f1da0069e00 0x7f1da0065f60 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1d94002f70 tx=0x7f1d94010bd0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.074 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da4a29700 1 --2- 172.21.15.196:0/4227192719 >> v2:172.21.15.80:3300/0 conn(0x7f1da0069e00 0x7f1da0065f60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1d94002f70 tx=0x7f1d94010bd0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.074 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da4a29700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1da0069e00 0x7f1da0065f60 secure :-1 s=READY pgs=78 cs=0 l=1 rx=0x7f1d94002f70 tx=0x7f1d94010bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:34.074 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da4a29700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1da0069e00 0x7f1da0065f60 secure :-1 s=THROTTLE_DONE pgs=78 cs=0 l=1 rx=0x7f1d94002f70 tx=0x7f1d94010bd0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.074 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1d9dffb700 1 -- 172.21.15.196:0/4227192719 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d94007550 con 0x7f1da0069e00 2020-05-18T21:40:34.075 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da4a29700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1da0069e00 0x7f1da0065f60 secure :-1 s=THROTTLE_DONE pgs=78 cs=0 l=1 rx=0x7f1d94002f70 tx=0x7f1d94010bd0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.075 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da4a29700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1da0069e00 0x7f1da0065f60 secure :-1 s=THROTTLE_DONE pgs=78 cs=0 l=1 rx=0x7f1d94002f70 tx=0x7f1d94010bd0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.075 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1d9dffb700 1 -- 172.21.15.196:0/4227192719 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d94005700 con 0x7f1da0069e00 2020-05-18T21:40:34.075 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da5a2b700 1 -- 172.21.15.196:0/4227192719 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da00a3fb0 con 0x7f1da0069e00 2020-05-18T21:40:34.075 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1d9dffb700 1 -- 172.21.15.196:0/4227192719 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d94005a60 con 0x7f1da0069e00 2020-05-18T21:40:34.076 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da5a2b700 1 -- 172.21.15.196:0/4227192719 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da00a3600 con 0x7f1da0069e00 2020-05-18T21:40:34.076 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da4a29700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1da0069e00 0x7f1da0065f60 secure :-1 s=THROTTLE_DONE pgs=78 cs=0 l=1 rx=0x7f1d94002f70 tx=0x7f1d94010bd0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.076 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da5a2b700 1 -- 172.21.15.196:0/4227192719 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da00a0270 con 0x7f1da0069e00 2020-05-18T21:40:34.076 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1d9dffb700 1 -- 172.21.15.196:0/4227192719 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f1d94005c60 con 0x7f1da0069e00 2020-05-18T21:40:34.077 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1da4a29700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1da0069e00 0x7f1da0065f60 secure :-1 s=THROTTLE_DONE pgs=78 cs=0 l=1 rx=0x7f1d94002f70 tx=0x7f1d94010bd0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.077 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1d9dffb700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1d8804fd10 0x7f1d880521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:34.077 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1d9dffb700 1 -- 172.21.15.196:0/4227192719 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1d94063cf0 con 0x7f1da0069e00 2020-05-18T21:40:34.078 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.063+0000 7f1d9ffff700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1d8804fd10 0x7f1d880521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.078 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.067+0000 7f1d9ffff700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1d8804fd10 0x7f1d880521d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1d8c000bc0 tx=0x7f1d8c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.078 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.067+0000 7f1d9ffff700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1d8804fd10 0x7f1d880521d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1d8c000bc0 tx=0x7f1d8c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.078 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.067+0000 7f1d9ffff700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1d8804fd10 0x7f1d880521d0 secure :-1 s=READY pgs=38 cs=0 l=1 rx=0x7f1d8c000bc0 tx=0x7f1d8c007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:34.079 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.071+0000 7f1da4a29700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1da0069e00 0x7f1da0065f60 secure :-1 s=THROTTLE_DONE pgs=78 cs=0 l=1 rx=0x7f1d94002f70 tx=0x7f1d94010bd0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.079 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.071+0000 7f1d9dffb700 1 -- 172.21.15.196:0/4227192719 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f1d9403e0e0 con 0x7f1da0069e00 2020-05-18T21:40:34.221 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1da5a2b700 1 -- 172.21.15.196:0/4227192719 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1da00569e0 con 0x7f1da0069e00 2020-05-18T21:40:34.222 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1da4a29700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1da0069e00 0x7f1da0065f60 secure :-1 s=THROTTLE_DONE pgs=78 cs=0 l=1 rx=0x7f1d94002f70 tx=0x7f1d94010bd0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.223 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1d9dffb700 1 -- 172.21.15.196:0/4227192719 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+1420 (secure 0 0 0) 0x7f1d94041700 con 0x7f1da0069e00 2020-05-18T21:40:34.223 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:34.223 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":3,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-05-18T21:40:32.208271+0000","modified":"2020-05-18T21:40:33.915142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-05-18T21:40:34.224 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1da5a2b700 1 -- 172.21.15.196:0/4227192719 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1d8804fd10 msgr2=0x7f1d880521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.224 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1da5a2b700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1d8804fd10 0x7f1d880521d0 secure :-1 s=READY pgs=38 cs=0 l=1 rx=0x7f1d8c000bc0 tx=0x7f1d8c007a80).stop 2020-05-18T21:40:34.225 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1da5a2b700 1 -- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1da0069e00 msgr2=0x7f1da0065f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.225 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1da5a2b700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1da0069e00 0x7f1da0065f60 secure :-1 s=READY pgs=78 cs=0 l=1 rx=0x7f1d94002f70 tx=0x7f1d94010bd0).stop 2020-05-18T21:40:34.225 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1da5a2b700 1 -- 172.21.15.196:0/4227192719 shutdown_connections 2020-05-18T21:40:34.225 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1da5a2b700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1d8804fd10 0x7f1d880521d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.226 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1da5a2b700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1da0132ea0 0x7f1da009eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.226 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1da5a2b700 1 --2- 172.21.15.196:0/4227192719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1da0069e00 0x7f1da0065f60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.226 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1da5a2b700 1 -- 172.21.15.196:0/4227192719 >> 172.21.15.196:0/4227192719 conn(0x7f1da0060e20 msgr2=0x7f1da005eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:34.226 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1da5a2b700 1 -- 172.21.15.196:0/4227192719 shutdown_connections 2020-05-18T21:40:34.226 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.223+0000 7f1da5a2b700 1 -- 172.21.15.196:0/4227192719 wait complete. 2020-05-18T21:40:34.227 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 3 2020-05-18T21:40:34.235 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-05-18T21:40:34.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d41427700 1 Processor -- start 2020-05-18T21:40:34.339 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d41427700 1 -- start start 2020-05-18T21:40:34.341 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d41427700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6d3c0a3e40 0x7f6d3c0a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:34.341 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d41427700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6d3c0a7320 0x7f6d3c0a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:34.342 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d41427700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d3c0a4850 con 0x7f6d3c0a5f80 2020-05-18T21:40:34.342 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d41427700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3c0a5890 con 0x7f6d3c0a7320 2020-05-18T21:40:34.342 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d41427700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c09dff0 con 0x7f6d3c0a3e40 2020-05-18T21:40:34.342 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d3b7fe700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f6d3c0a5f80 0x7f6d3c0a6c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43100/0 (socket says 172.21.15.196:43100) 2020-05-18T21:40:34.342 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d3b7fe700 1 -- 172.21.15.196:0/466820617 learned_addr learned my addr 172.21.15.196:0/466820617 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:34.343 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d40c26700 1 --2- 172.21.15.196:0/466820617 >> v2:172.21.15.196:3300/0 conn(0x7f6d3c0a7320 0x7f6d3c0a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.343 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d3bfff700 1 --2- 172.21.15.196:0/466820617 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6d3c0a3e40 0x7f6d3c0a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.343 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d3affd700 1 -- 172.21.15.196:0/466820617 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061856413 0 0) 0x7f6d3c0a4850 con 0x7f6d3c0a5f80 2020-05-18T21:40:34.343 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d3affd700 1 -- 172.21.15.196:0/466820617 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d1c003270 con 0x7f6d3c0a5f80 2020-05-18T21:40:34.343 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d40c26700 1 -- 172.21.15.196:0/466820617 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6d3c0a3e40 msgr2=0x7f6d3c0a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:34.344 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d40c26700 1 --2- 172.21.15.196:0/466820617 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6d3c0a3e40 0x7f6d3c0a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:34.344 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d40c26700 1 -- 172.21.15.196:0/466820617 >> v1:172.21.15.196:6789/0 conn(0x7f6d3c0a5f80 legacy=0x7f6d3c0a6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.344 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d40c26700 1 -- 172.21.15.196:0/466820617 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c0a63e0 con 0x7f6d3c0a7320 2020-05-18T21:40:34.344 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d40c26700 1 --2- 172.21.15.196:0/466820617 >> v2:172.21.15.196:3300/0 conn(0x7f6d3c0a7320 0x7f6d3c0a77b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d40c26700 1 --2- 172.21.15.196:0/466820617 >> v2:172.21.15.196:3300/0 conn(0x7f6d3c0a7320 0x7f6d3c0a77b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d40c26700 1 --2- 172.21.15.196:0/466820617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c0a77b0 secure :-1 s=READY pgs=99 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).ready entity=mon.1 client_cookie=3ebfa28eb394ee00 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:34.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d40c26700 1 --2- 172.21.15.196:0/466820617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c0a77b0 secure :-1 s=THROTTLE_DONE pgs=99 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d3affd700 1 -- 172.21.15.196:0/466820617 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d34011db0 con 0x7f6d3c0a7320 2020-05-18T21:40:34.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d40c26700 1 --2- 172.21.15.196:0/466820617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c0a77b0 secure :-1 s=THROTTLE_DONE pgs=99 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d3affd700 1 -- 172.21.15.196:0/466820617 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3400a180 con 0x7f6d3c0a7320 2020-05-18T21:40:34.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d40c26700 1 --2- 172.21.15.196:0/466820617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c0a77b0 secure :-1 s=THROTTLE_DONE pgs=99 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d3affd700 1 -- 172.21.15.196:0/466820617 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d3400a560 con 0x7f6d3c0a7320 2020-05-18T21:40:34.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d41427700 1 -- 172.21.15.196:0/466820617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 msgr2=0x7f6d3c0a77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.339+0000 7f6d41427700 1 --2- 172.21.15.196:0/466820617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c0a77b0 secure :-1 s=READY pgs=99 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).stop 2020-05-18T21:40:34.350 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.343+0000 7f6d41427700 1 -- 172.21.15.196:0/466820617 shutdown_connections 2020-05-18T21:40:34.350 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.343+0000 7f6d41427700 1 --2- 172.21.15.196:0/466820617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c0a77b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.350 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.343+0000 7f6d41427700 1 --2- 172.21.15.196:0/466820617 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6d3c0a3e40 0x7f6d3c0a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:34.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.343+0000 7f6d41427700 1 -- 172.21.15.196:0/466820617 >> 172.21.15.196:0/466820617 conn(0x7f6d3c05a7c0 msgr2=0x7f6d3c05abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:34.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.343+0000 7f6d41427700 1 -- 172.21.15.196:0/466820617 shutdown_connections 2020-05-18T21:40:34.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.343+0000 7f6d41427700 1 -- 172.21.15.196:0/466820617 wait complete. 2020-05-18T21:40:34.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.343+0000 7f6d41427700 1 Processor -- start 2020-05-18T21:40:34.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.343+0000 7f6d41427700 1 -- start start 2020-05-18T21:40:34.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d41427700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6d3c0a5f80 0x7f6d3c0a9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:34.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d41427700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:34.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d41427700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d3c08f740 con 0x7f6d3c0a3e40 2020-05-18T21:40:34.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d41427700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3c08f9e0 con 0x7f6d3c0a5f80 2020-05-18T21:40:34.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d41427700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3c13b770 con 0x7f6d3c0a7320 2020-05-18T21:40:34.353 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d40c26700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.353 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d3bfff700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f6d3c0a3e40 0x7f6d3c064960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43106/0 (socket says 172.21.15.196:43106) 2020-05-18T21:40:34.353 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d3bfff700 1 -- 172.21.15.196:0/390707331 learned_addr learned my addr 172.21.15.196:0/390707331 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:34.353 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d3b7fe700 1 --2- 172.21.15.196:0/390707331 >> v2:172.21.15.80:3300/0 conn(0x7f6d3c0a5f80 0x7f6d3c0a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.353 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d397fa700 1 -- 172.21.15.196:0/390707331 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451087394 0 0) 0x7f6d3c08f740 con 0x7f6d3c0a3e40 2020-05-18T21:40:34.354 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d397fa700 1 -- 172.21.15.196:0/390707331 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d20002fe0 con 0x7f6d3c0a3e40 2020-05-18T21:40:34.354 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d40c26700 1 -- 172.21.15.196:0/390707331 >> v2:172.21.15.80:3300/0 conn(0x7f6d3c0a5f80 msgr2=0x7f6d3c0a9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.354 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d40c26700 1 --2- 172.21.15.196:0/390707331 >> v2:172.21.15.80:3300/0 conn(0x7f6d3c0a5f80 0x7f6d3c0a9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.354 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d40c26700 1 -- 172.21.15.196:0/390707331 >> v1:172.21.15.196:6789/0 conn(0x7f6d3c0a3e40 legacy=0x7f6d3c064960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d40c26700 1 -- 172.21.15.196:0/390707331 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c0659f0 con 0x7f6d3c0a7320 2020-05-18T21:40:34.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d40c26700 1 --2- 172.21.15.196:0/390707331 >> v2:172.21.15.196:3300/0 conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d40c26700 1 --2- 172.21.15.196:0/390707331 >> v2:172.21.15.196:3300/0 conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d40c26700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 secure :-1 s=READY pgs=101 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:34.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d40c26700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.356 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d397fa700 1 -- 172.21.15.196:0/390707331 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d34011e90 con 0x7f6d3c0a7320 2020-05-18T21:40:34.356 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d40c26700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.356 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d397fa700 1 -- 172.21.15.196:0/390707331 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d34005c70 con 0x7f6d3c0a7320 2020-05-18T21:40:34.357 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d40c26700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.357 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d41427700 1 -- 172.21.15.196:0/390707331 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3c0a5b80 con 0x7f6d3c0a7320 2020-05-18T21:40:34.357 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d397fa700 1 -- 172.21.15.196:0/390707331 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d34005de0 con 0x7f6d3c0a7320 2020-05-18T21:40:34.357 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d41427700 1 -- 172.21.15.196:0/390707331 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d3c065070 con 0x7f6d3c0a7320 2020-05-18T21:40:34.357 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d40c26700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.358 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d41427700 1 -- 172.21.15.196:0/390707331 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3c0569e0 con 0x7f6d3c0a7320 2020-05-18T21:40:34.358 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d397fa700 1 -- 172.21.15.196:0/390707331 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f6d34007360 con 0x7f6d3c0a7320 2020-05-18T21:40:34.358 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.347+0000 7f6d40c26700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.358 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.351+0000 7f6d397fa700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6d2004fd30 0x7f6d200521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:34.358 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.351+0000 7f6d397fa700 1 -- 172.21.15.196:0/390707331 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6d340649d0 con 0x7f6d3c0a7320 2020-05-18T21:40:34.359 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.351+0000 7f6d3bfff700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6d2004fd30 0x7f6d200521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.359 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.351+0000 7f6d3bfff700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6d2004fd30 0x7f6d200521f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f6d24002f70 tx=0x7f6d240090f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.359 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.351+0000 7f6d3bfff700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6d2004fd30 0x7f6d200521f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f6d24002f70 tx=0x7f6d240090f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.359 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.351+0000 7f6d3bfff700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6d2004fd30 0x7f6d200521f0 secure :-1 s=READY pgs=39 cs=0 l=1 rx=0x7f6d24002f70 tx=0x7f6d240090f0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:34.360 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.355+0000 7f6d40c26700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.360 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.355+0000 7f6d397fa700 1 -- 172.21.15.196:0/390707331 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f6d3403ee80 con 0x7f6d3c0a7320 2020-05-18T21:40:34.470 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.471+0000 7f6d41427700 1 -- 172.21.15.196:0/390707331 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d3c08fb20 con 0x7f6d3c0a7320 2020-05-18T21:40:34.472 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.471+0000 7f6d40c26700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.473 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.471+0000 7f6d397fa700 1 -- 172.21.15.196:0/390707331 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16089 (secure 0 0 0) 0x7f6d340424a0 con 0x7f6d3c0a7320 2020-05-18T21:40:34.473 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:34.473 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"fsid":"d048673a-3d91-4cc6-882d-e9cc70e7cd6c","created":"2020-05-18T21:40:15.911738+0000","modified":"2020-05-18T21:40:33.211283+0000","last_up_change":"2020-05-18T21:40:22.762927+0000","last_in_change":"2020-05-18T21:40:22.762927+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-05-18T21:40:19.808941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-05-18T21:40:26.944311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-05-18T21:40:30.135661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-05-18T21:40:31.183488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8db63a0a-61dc-42b5-9559-fddcf436cab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":12771},{"type":"v1","addr":"172.21.15.196:6801","nonce":12771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":12771},{"type":"v1","addr":"172.21.15.196:6803","nonce":12771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":12771},{"type":"v1","addr":"172.21.15.196:6807","nonce":12771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":12771},{"type":"v1","addr":"172.21.15.196:6805","nonce":12771}]},"public_addr":"172.21.15.196:6801/12771","cluster_addr":"172.21.15.196:6803/12771","heartbeat_back_addr":"172.21.15.196:6807/12771","heartbeat_front_addr":"172.21.15.196:6805/12771","state":["exists","up"]},{"osd":1,"uuid":"6773b0c3-9f49-42a2-b75b-c10c734ff086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":12772},{"type":"v1","addr":"172.21.15.196:6817","nonce":12772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":12772},{"type":"v1","addr":"172.21.15.196:6819","nonce":12772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":12772},{"type":"v1","addr":"172.21.15.196:6823","nonce":12772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":12772},{"type":"v1","addr":"172.21.15.196:6821","nonce":12772}]},"public_addr":"172.21.15.196:6817/12772","cluster_addr":"172.21.15.196:6819/12772","heartbeat_back_addr":"172.21.15.196:6823/12772","heartbeat_front_addr":"172.21.15.196:6821/12772","state":["exists","up"]},{"osd":2,"uuid":"ac1f2e05-9bb7-4768-b712-10e4d2ea4154","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":12773},{"type":"v1","addr":"172.21.15.196:6809","nonce":12773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":12773},{"type":"v1","addr":"172.21.15.196:6811","nonce":12773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":12773},{"type":"v1","addr":"172.21.15.196:6815","nonce":12773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":12773},{"type":"v1","addr":"172.21.15.196:6813","nonce":12773}]},"public_addr":"172.21.15.196:6809/12773","cluster_addr":"172.21.15.196:6811/12773","heartbeat_back_addr":"172.21.15.196:6815/12773","heartbeat_front_addr":"172.21.15.196:6813/12773","state":["exists","up"]},{"osd":3,"uuid":"bae5ab40-235b-4e0f-8d0d-524afd099b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":12775},{"type":"v1","addr":"172.21.15.196:6825","nonce":12775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":12775},{"type":"v1","addr":"172.21.15.196:6827","nonce":12775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":12775},{"type":"v1","addr":"172.21.15.196:6831","nonce":12775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":12775},{"type":"v1","addr":"172.21.15.196:6829","nonce":12775}]},"public_addr":"172.21.15.196:6825/12775","cluster_addr":"172.21.15.196:6827/12775","heartbeat_back_addr":"172.21.15.196:6831/12775","heartbeat_front_addr":"172.21.15.196:6829/12775","state":["exists","up"]},{"osd":4,"uuid":"275fc52c-58e4-4f35-a0cb-e117fe36bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":13040},{"type":"v1","addr":"172.21.15.80:6819","nonce":13040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6821","nonce":13040},{"type":"v1","addr":"172.21.15.80:6823","nonce":13040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6829","nonce":13040},{"type":"v1","addr":"172.21.15.80:6831","nonce":13040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":13040},{"type":"v1","addr":"172.21.15.80:6827","nonce":13040}]},"public_addr":"172.21.15.80:6819/13040","cluster_addr":"172.21.15.80:6823/13040","heartbeat_back_addr":"172.21.15.80:6831/13040","heartbeat_front_addr":"172.21.15.80:6827/13040","state":["exists","up"]},{"osd":5,"uuid":"9795b89c-e2f8-4abf-b9c7-796727a57002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":13037},{"type":"v1","addr":"172.21.15.80:6818","nonce":13037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":13037},{"type":"v1","addr":"172.21.15.80:6822","nonce":13037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":13037},{"type":"v1","addr":"172.21.15.80:6830","nonce":13037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":13037},{"type":"v1","addr":"172.21.15.80:6826","nonce":13037}]},"public_addr":"172.21.15.80:6818/13037","cluster_addr":"172.21.15.80:6822/13037","heartbeat_back_addr":"172.21.15.80:6830/13037","heartbeat_front_addr":"172.21.15.80:6826/13037","state":["exists","up"]},{"osd":6,"uuid":"e42df2e8-9cc5-4c7e-a243-bd18ccfbd002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":13039},{"type":"v1","addr":"172.21.15.80:6804","nonce":13039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":13039},{"type":"v1","addr":"172.21.15.80:6808","nonce":13039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":13039},{"type":"v1","addr":"172.21.15.80:6813","nonce":13039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":13039},{"type":"v1","addr":"172.21.15.80:6810","nonce":13039}]},"public_addr":"172.21.15.80:6804/13039","cluster_addr":"172.21.15.80:6808/13039","heartbeat_back_addr":"172.21.15.80:6813/13039","heartbeat_front_addr":"172.21.15.80:6810/13039","state":["exists","up"]},{"osd":7,"uuid":"ebd6c938-beff-4668-b70e-076ed22bb8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":13038},{"type":"v1","addr":"172.21.15.80:6801","nonce":13038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":13038},{"type":"v1","addr":"172.21.15.80:6805","nonce":13038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":13038},{"type":"v1","addr":"172.21.15.80:6817","nonce":13038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":13038},{"type":"v1","addr":"172.21.15.80:6811","nonce":13038}]},"public_addr":"172.21.15.80:6801/13038","cluster_addr":"172.21.15.80:6805/13038","heartbeat_back_addr":"172.21.15.80:6817/13038","heartbeat_front_addr":"172.21.15.80:6811/13038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.134515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.439836+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.476825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.176642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.452301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.427183+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.230250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.210168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-18T21:40:34.475 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.475+0000 7f6d41427700 1 -- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6d2004fd30 msgr2=0x7f6d200521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.476 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.475+0000 7f6d41427700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6d2004fd30 0x7f6d200521f0 secure :-1 s=READY pgs=39 cs=0 l=1 rx=0x7f6d24002f70 tx=0x7f6d240090f0).stop 2020-05-18T21:40:34.476 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.475+0000 7f6d41427700 1 -- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 msgr2=0x7f6d3c13a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.476 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.475+0000 7f6d41427700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 secure :-1 s=READY pgs=101 cs=0 l=1 rx=0x7f6d340032f0 tx=0x7f6d34007e00).stop 2020-05-18T21:40:34.476 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.475+0000 7f6d41427700 1 -- 172.21.15.196:0/390707331 shutdown_connections 2020-05-18T21:40:34.476 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.475+0000 7f6d41427700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6d2004fd30 0x7f6d200521f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.477 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.475+0000 7f6d41427700 1 --2- 172.21.15.196:0/390707331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d3c0a7320 0x7f6d3c13a9f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.477 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.475+0000 7f6d41427700 1 --2- 172.21.15.196:0/390707331 >> v2:172.21.15.80:3300/0 conn(0x7f6d3c0a5f80 0x7f6d3c0a9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.477 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.475+0000 7f6d41427700 1 -- 172.21.15.196:0/390707331 >> 172.21.15.196:0/390707331 conn(0x7f6d3c05a7c0 msgr2=0x7f6d3c05eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:34.477 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.475+0000 7f6d41427700 1 -- 172.21.15.196:0/390707331 shutdown_connections 2020-05-18T21:40:34.478 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.475+0000 7f6d41427700 1 -- 172.21.15.196:0/390707331 wait complete. 2020-05-18T21:40:34.491 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-05-18T21:40:34.492 INFO:tasks.ceph.mds.a:Restarting daemon 2020-05-18T21:40:34.492 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-05-18T21:40:34.495 INFO:tasks.ceph.mds.a:Started 2020-05-18T21:40:34.495 INFO:tasks.ceph.mds.b:Restarting daemon 2020-05-18T21:40:34.496 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-05-18T21:40:34.498 INFO:tasks.ceph.mds.b:Started 2020-05-18T21:40:34.499 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-05-18T21:40:34.500 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-05-18T21:40:34.566 INFO:tasks.ceph.mds.a.smithi196.stdout:starting mds.a at 2020-05-18T21:40:34.575 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2020-05-18T21:40:34.596 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.595+0000 7f01016e8700 1 Processor -- start 2020-05-18T21:40:34.596 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.595+0000 7f01016e8700 1 -- start start 2020-05-18T21:40:34.599 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.595+0000 7f01016e8700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f00fc0ab910 0x7f00fc0a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:34.599 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.595+0000 7f01016e8700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc137ff0 con 0x7f00fc137bb0 2020-05-18T21:40:34.599 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.595+0000 7f01016e8700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc138250 con 0x7f00fc134cf0 2020-05-18T21:40:34.599 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.595+0000 7f01016e8700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00fc0a66d0 con 0x7f00fc0ab910 2020-05-18T21:40:34.599 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.595+0000 7f00fb7fe700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f00fc134cf0 0x7f00fc12e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43132/0 (socket says 172.21.15.196:43132) 2020-05-18T21:40:34.600 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.595+0000 7f00fb7fe700 1 -- 172.21.15.196:0/2544302278 learned_addr learned my addr 172.21.15.196:0/2544302278 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:34.600 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.595+0000 7f00faffd700 1 --2- 172.21.15.196:0/2544302278 >> v2:172.21.15.196:3300/0 conn(0x7f00fc0ab910 0x7f00fc0a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.600 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f00faffd700 1 -- 172.21.15.196:0/2544302278 >> v1:172.21.15.196:6789/0 conn(0x7f00fc134cf0 legacy=0x7f00fc12e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.601 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f00faffd700 1 -- 172.21.15.196:0/2544302278 >> v1:172.21.15.80:6789/0 conn(0x7f00fc137bb0 legacy=0x7f00fc1345e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.601 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f00f9ffb700 1 -- 172.21.15.196:0/2544302278 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116128322 0 0) 0x7f00fc138250 con 0x7f00fc134cf0 2020-05-18T21:40:34.601 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f00faffd700 1 -- 172.21.15.196:0/2544302278 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc1384f0 con 0x7f00fc0ab910 2020-05-18T21:40:34.601 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f00faffd700 1 --2- 172.21.15.196:0/2544302278 >> v2:172.21.15.196:3300/0 conn(0x7f00fc0ab910 0x7f00fc0a5fd0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f00ec001110 tx=0x7f00f0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.602 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f00faffd700 1 --2- 172.21.15.196:0/2544302278 >> v2:172.21.15.196:3300/0 conn(0x7f00fc0ab910 0x7f00fc0a5fd0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f00ec001110 tx=0x7f00f0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.602 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f00faffd700 1 --2- 172.21.15.196:0/2544302278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00fc0ab910 0x7f00fc0a5fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rx=0x7f00ec001110 tx=0x7f00f0007a80).ready entity=mon.1 client_cookie=2adf9f04d303f413 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:34.602 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f00faffd700 1 --2- 172.21.15.196:0/2544302278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00fc0ab910 0x7f00fc0a5fd0 secure :-1 s=THROTTLE_DONE pgs=108 cs=0 l=1 rx=0x7f00ec001110 tx=0x7f00f0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.602 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f00faffd700 1 --2- 172.21.15.196:0/2544302278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00fc0ab910 0x7f00fc0a5fd0 secure :-1 s=THROTTLE_DONE pgs=108 cs=0 l=1 rx=0x7f00ec001110 tx=0x7f00f0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.602 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f00f9ffb700 1 -- 172.21.15.196:0/2544302278 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00f0008e40 con 0x7f00fc0ab910 2020-05-18T21:40:34.603 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f00faffd700 1 --2- 172.21.15.196:0/2544302278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00fc0ab910 0x7f00fc0a5fd0 secure :-1 s=THROTTLE_DONE pgs=108 cs=0 l=1 rx=0x7f00ec001110 tx=0x7f00f0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.603 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f00f9ffb700 1 -- 172.21.15.196:0/2544302278 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f00115a0 con 0x7f00fc0ab910 2020-05-18T21:40:34.603 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f00f9ffb700 1 -- 172.21.15.196:0/2544302278 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00f0011980 con 0x7f00fc0ab910 2020-05-18T21:40:34.603 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f01016e8700 1 -- 172.21.15.196:0/2544302278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00fc0ab910 msgr2=0x7f00fc0a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.603 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f01016e8700 1 --2- 172.21.15.196:0/2544302278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00fc0ab910 0x7f00fc0a5fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rx=0x7f00ec001110 tx=0x7f00f0007a80).stop 2020-05-18T21:40:34.604 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f01016e8700 1 -- 172.21.15.196:0/2544302278 shutdown_connections 2020-05-18T21:40:34.604 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f01016e8700 1 --2- 172.21.15.196:0/2544302278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00fc0ab910 0x7f00fc0a5fd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.604 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f01016e8700 1 -- 172.21.15.196:0/2544302278 >> 172.21.15.196:0/2544302278 conn(0x7f00fc060fe0 msgr2=0x7f00fc0aaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:34.604 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f01016e8700 1 -- 172.21.15.196:0/2544302278 shutdown_connections 2020-05-18T21:40:34.604 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f01016e8700 1 -- 172.21.15.196:0/2544302278 wait complete. 2020-05-18T21:40:34.608 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f01016e8700 1 Processor -- start 2020-05-18T21:40:34.608 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.599+0000 7f01016e8700 1 -- start start 2020-05-18T21:40:34.608 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f01016e8700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f00fc134cf0 0x7f00fc061b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:34.608 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f01016e8700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00fc137bb0 0x7f00fc14c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:34.608 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f01016e8700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc1357b0 con 0x7f00fc0ab910 2020-05-18T21:40:34.609 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f01016e8700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00fc066b50 con 0x7f00fc134cf0 2020-05-18T21:40:34.609 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00faffd700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f00fc0ab910 0x7f00fc0a3130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43138/0 (socket says 172.21.15.196:43138) 2020-05-18T21:40:34.609 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f01016e8700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0a1590 con 0x7f00fc137bb0 2020-05-18T21:40:34.609 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00faffd700 1 -- 172.21.15.196:0/468341403 learned_addr learned my addr 172.21.15.196:0/468341403 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:34.609 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00fa7fc700 1 --2- 172.21.15.196:0/468341403 >> v2:172.21.15.80:3300/0 conn(0x7f00fc134cf0 0x7f00fc061b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.610 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00fb7fe700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00fc137bb0 0x7f00fc14c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.610 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00ebfff700 1 -- 172.21.15.196:0/468341403 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837538563 0 0) 0x7f00fc1357b0 con 0x7f00fc0ab910 2020-05-18T21:40:34.610 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00ebfff700 1 -- 172.21.15.196:0/468341403 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d8002fe0 con 0x7f00fc0ab910 2020-05-18T21:40:34.610 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00fa7fc700 1 -- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00fc137bb0 msgr2=0x7f00fc14c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.610 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00fa7fc700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00fc137bb0 0x7f00fc14c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.611 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00fa7fc700 1 -- 172.21.15.196:0/468341403 >> v1:172.21.15.196:6789/0 conn(0x7f00fc0ab910 legacy=0x7f00fc0a3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.611 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00fa7fc700 1 -- 172.21.15.196:0/468341403 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc0a4ec0 con 0x7f00fc134cf0 2020-05-18T21:40:34.611 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00fa7fc700 1 --2- 172.21.15.196:0/468341403 >> v2:172.21.15.80:3300/0 conn(0x7f00fc134cf0 0x7f00fc061b10 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f00fc0a5200 tx=0x7f00ec00c4d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.611 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00fa7fc700 1 --2- 172.21.15.196:0/468341403 >> v2:172.21.15.80:3300/0 conn(0x7f00fc134cf0 0x7f00fc061b10 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f00fc0a5200 tx=0x7f00ec00c4d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.612 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00fa7fc700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00fc134cf0 0x7f00fc061b10 secure :-1 s=READY pgs=84 cs=0 l=1 rx=0x7f00fc0a5200 tx=0x7f00ec00c4d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:34.612 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00fa7fc700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00fc134cf0 0x7f00fc061b10 secure :-1 s=THROTTLE_DONE pgs=84 cs=0 l=1 rx=0x7f00fc0a5200 tx=0x7f00ec00c4d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.612 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00ebfff700 1 -- 172.21.15.196:0/468341403 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00ec00cc40 con 0x7f00fc134cf0 2020-05-18T21:40:34.612 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00fa7fc700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00fc134cf0 0x7f00fc061b10 secure :-1 s=THROTTLE_DONE pgs=84 cs=0 l=1 rx=0x7f00fc0a5200 tx=0x7f00ec00c4d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.612 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00fa7fc700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00fc134cf0 0x7f00fc061b10 secure :-1 s=THROTTLE_DONE pgs=84 cs=0 l=1 rx=0x7f00fc0a5200 tx=0x7f00ec00c4d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.613 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00ebfff700 1 -- 172.21.15.196:0/468341403 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec00cda0 con 0x7f00fc134cf0 2020-05-18T21:40:34.613 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f00ebfff700 1 -- 172.21.15.196:0/468341403 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00ec011140 con 0x7f00fc134cf0 2020-05-18T21:40:34.613 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f01016e8700 1 -- 172.21.15.196:0/468341403 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc129870 con 0x7f00fc134cf0 2020-05-18T21:40:34.613 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.603+0000 7f01016e8700 1 -- 172.21.15.196:0/468341403 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc0a2be0 con 0x7f00fc134cf0 2020-05-18T21:40:34.613 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.607+0000 7f00fa7fc700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00fc134cf0 0x7f00fc061b10 secure :-1 s=THROTTLE_DONE pgs=84 cs=0 l=1 rx=0x7f00fc0a5200 tx=0x7f00ec00c4d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.607+0000 7f01016e8700 1 -- 172.21.15.196:0/468341403 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc129a70 con 0x7f00fc134cf0 2020-05-18T21:40:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.607+0000 7f00ebfff700 1 -- 172.21.15.196:0/468341403 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f00ec006520 con 0x7f00fc134cf0 2020-05-18T21:40:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.607+0000 7f00fa7fc700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00fc134cf0 0x7f00fc061b10 secure :-1 s=THROTTLE_DONE pgs=84 cs=0 l=1 rx=0x7f00fc0a5200 tx=0x7f00ec00c4d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.607+0000 7f00ebfff700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f00d804fd90 0x7f00d8052250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:34.614 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.607+0000 7f00ebfff700 1 -- 172.21.15.196:0/468341403 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f00ec063690 con 0x7f00fc134cf0 2020-05-18T21:40:34.615 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.607+0000 7f00faffd700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f00d804fd90 0x7f00d8052250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.615 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.607+0000 7f00faffd700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f00d804fd90 0x7f00d8052250 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f00f0000bc0 tx=0x7f00f0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.615 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.607+0000 7f00faffd700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f00d804fd90 0x7f00d8052250 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f00f0000bc0 tx=0x7f00f0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.615 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.607+0000 7f00faffd700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f00d804fd90 0x7f00d8052250 secure :-1 s=READY pgs=40 cs=0 l=1 rx=0x7f00f0000bc0 tx=0x7f00f0007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:34.616 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.615+0000 7f00fa7fc700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00fc134cf0 0x7f00fc061b10 secure :-1 s=THROTTLE_DONE pgs=84 cs=0 l=1 rx=0x7f00fc0a5200 tx=0x7f00ec00c4d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.616 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.615+0000 7f00ebfff700 1 -- 172.21.15.196:0/468341403 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f00ec03d9c0 con 0x7f00fc134cf0 2020-05-18T21:40:34.733 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.735+0000 7f01016e8700 1 -- 172.21.15.196:0/468341403 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00fc0569e0 con 0x7f00fc134cf0 2020-05-18T21:40:34.735 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.735+0000 7f00fa7fc700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00fc134cf0 0x7f00fc061b10 secure :-1 s=THROTTLE_DONE pgs=84 cs=0 l=1 rx=0x7f00fc0a5200 tx=0x7f00ec00c4d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.736 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.735+0000 7f00ebfff700 1 -- 172.21.15.196:0/468341403 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16089 (secure 0 0 0) 0x7f00ec040fe0 con 0x7f00fc134cf0 2020-05-18T21:40:34.736 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:34.737 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"fsid":"d048673a-3d91-4cc6-882d-e9cc70e7cd6c","created":"2020-05-18T21:40:15.911738+0000","modified":"2020-05-18T21:40:33.211283+0000","last_up_change":"2020-05-18T21:40:22.762927+0000","last_in_change":"2020-05-18T21:40:22.762927+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-05-18T21:40:19.808941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-05-18T21:40:26.944311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-05-18T21:40:30.135661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-05-18T21:40:31.183488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8db63a0a-61dc-42b5-9559-fddcf436cab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":12771},{"type":"v1","addr":"172.21.15.196:6801","nonce":12771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":12771},{"type":"v1","addr":"172.21.15.196:6803","nonce":12771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":12771},{"type":"v1","addr":"172.21.15.196:6807","nonce":12771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":12771},{"type":"v1","addr":"172.21.15.196:6805","nonce":12771}]},"public_addr":"172.21.15.196:6801/12771","cluster_addr":"172.21.15.196:6803/12771","heartbeat_back_addr":"172.21.15.196:6807/12771","heartbeat_front_addr":"172.21.15.196:6805/12771","state":["exists","up"]},{"osd":1,"uuid":"6773b0c3-9f49-42a2-b75b-c10c734ff086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":12772},{"type":"v1","addr":"172.21.15.196:6817","nonce":12772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":12772},{"type":"v1","addr":"172.21.15.196:6819","nonce":12772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":12772},{"type":"v1","addr":"172.21.15.196:6823","nonce":12772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":12772},{"type":"v1","addr":"172.21.15.196:6821","nonce":12772}]},"public_addr":"172.21.15.196:6817/12772","cluster_addr":"172.21.15.196:6819/12772","heartbeat_back_addr":"172.21.15.196:6823/12772","heartbeat_front_addr":"172.21.15.196:6821/12772","state":["exists","up"]},{"osd":2,"uuid":"ac1f2e05-9bb7-4768-b712-10e4d2ea4154","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":12773},{"type":"v1","addr":"172.21.15.196:6809","nonce":12773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":12773},{"type":"v1","addr":"172.21.15.196:6811","nonce":12773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":12773},{"type":"v1","addr":"172.21.15.196:6815","nonce":12773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":12773},{"type":"v1","addr":"172.21.15.196:6813","nonce":12773}]},"public_addr":"172.21.15.196:6809/12773","cluster_addr":"172.21.15.196:6811/12773","heartbeat_back_addr":"172.21.15.196:6815/12773","heartbeat_front_addr":"172.21.15.196:6813/12773","state":["exists","up"]},{"osd":3,"uuid":"bae5ab40-235b-4e0f-8d0d-524afd099b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":12775},{"type":"v1","addr":"172.21.15.196:6825","nonce":12775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":12775},{"type":"v1","addr":"172.21.15.196:6827","nonce":12775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":12775},{"type":"v1","addr":"172.21.15.196:6831","nonce":12775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":12775},{"type":"v1","addr":"172.21.15.196:6829","nonce":12775}]},"public_addr":"172.21.15.196:6825/12775","cluster_addr":"172.21.15.196:6827/12775","heartbeat_back_addr":"172.21.15.196:6831/12775","heartbeat_front_addr":"172.21.15.196:6829/12775","state":["exists","up"]},{"osd":4,"uuid":"275fc52c-58e4-4f35-a0cb-e117fe36bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":13040},{"type":"v1","addr":"172.21.15.80:6819","nonce":13040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6821","nonce":13040},{"type":"v1","addr":"172.21.15.80:6823","nonce":13040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6829","nonce":13040},{"type":"v1","addr":"172.21.15.80:6831","nonce":13040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":13040},{"type":"v1","addr":"172.21.15.80:6827","nonce":13040}]},"public_addr":"172.21.15.80:6819/13040","cluster_addr":"172.21.15.80:6823/13040","heartbeat_back_addr":"172.21.15.80:6831/13040","heartbeat_front_addr":"172.21.15.80:6827/13040","state":["exists","up"]},{"osd":5,"uuid":"9795b89c-e2f8-4abf-b9c7-796727a57002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":13037},{"type":"v1","addr":"172.21.15.80:6818","nonce":13037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":13037},{"type":"v1","addr":"172.21.15.80:6822","nonce":13037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":13037},{"type":"v1","addr":"172.21.15.80:6830","nonce":13037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":13037},{"type":"v1","addr":"172.21.15.80:6826","nonce":13037}]},"public_addr":"172.21.15.80:6818/13037","cluster_addr":"172.21.15.80:6822/13037","heartbeat_back_addr":"172.21.15.80:6830/13037","heartbeat_front_addr":"172.21.15.80:6826/13037","state":["exists","up"]},{"osd":6,"uuid":"e42df2e8-9cc5-4c7e-a243-bd18ccfbd002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":13039},{"type":"v1","addr":"172.21.15.80:6804","nonce":13039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":13039},{"type":"v1","addr":"172.21.15.80:6808","nonce":13039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":13039},{"type":"v1","addr":"172.21.15.80:6813","nonce":13039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":13039},{"type":"v1","addr":"172.21.15.80:6810","nonce":13039}]},"public_addr":"172.21.15.80:6804/13039","cluster_addr":"172.21.15.80:6808/13039","heartbeat_back_addr":"172.21.15.80:6813/13039","heartbeat_front_addr":"172.21.15.80:6810/13039","state":["exists","up"]},{"osd":7,"uuid":"ebd6c938-beff-4668-b70e-076ed22bb8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":13038},{"type":"v1","addr":"172.21.15.80:6801","nonce":13038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":13038},{"type":"v1","addr":"172.21.15.80:6805","nonce":13038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":13038},{"type":"v1","addr":"172.21.15.80:6817","nonce":13038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":13038},{"type":"v1","addr":"172.21.15.80:6811","nonce":13038}]},"public_addr":"172.21.15.80:6801/13038","cluster_addr":"172.21.15.80:6805/13038","heartbeat_back_addr":"172.21.15.80:6817/13038","heartbeat_front_addr":"172.21.15.80:6811/13038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.134515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.439836+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.476825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.176642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.452301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.427183+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.230250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.210168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-18T21:40:34.738 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.739+0000 7f01016e8700 1 -- 172.21.15.196:0/468341403 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f00d804fd90 msgr2=0x7f00d8052250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.738 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.739+0000 7f01016e8700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f00d804fd90 0x7f00d8052250 secure :-1 s=READY pgs=40 cs=0 l=1 rx=0x7f00f0000bc0 tx=0x7f00f0007a80).stop 2020-05-18T21:40:34.739 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.739+0000 7f01016e8700 1 -- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00fc134cf0 msgr2=0x7f00fc061b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.739 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.739+0000 7f01016e8700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00fc134cf0 0x7f00fc061b10 secure :-1 s=READY pgs=84 cs=0 l=1 rx=0x7f00fc0a5200 tx=0x7f00ec00c4d0).stop 2020-05-18T21:40:34.741 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.739+0000 7f01016e8700 1 -- 172.21.15.196:0/468341403 shutdown_connections 2020-05-18T21:40:34.741 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.739+0000 7f01016e8700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f00d804fd90 0x7f00d8052250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.741 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.739+0000 7f01016e8700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00fc137bb0 0x7f00fc14c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.741 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.739+0000 7f01016e8700 1 --2- 172.21.15.196:0/468341403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00fc134cf0 0x7f00fc061b10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.741 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.739+0000 7f01016e8700 1 -- 172.21.15.196:0/468341403 >> 172.21.15.196:0/468341403 conn(0x7f00fc060fe0 msgr2=0x7f00fc0a7940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:34.742 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.739+0000 7f01016e8700 1 -- 172.21.15.196:0/468341403 shutdown_connections 2020-05-18T21:40:34.742 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.739+0000 7f01016e8700 1 -- 172.21.15.196:0/468341403 wait complete. 2020-05-18T21:40:34.749 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-05-18T21:40:19.808941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-05-18T21:40:26.944311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-05-18T21:40:30.135661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-05-18T21:40:31.183488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-05-18T21:40:34.750 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-05-18T21:40:34.860 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.859+0000 7fa29a215700 1 Processor -- start 2020-05-18T21:40:34.861 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.859+0000 7fa29a215700 1 -- start start 2020-05-18T21:40:34.863 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa29a215700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa2940a1840 0x7fa2940a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:34.864 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa29a215700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa29409e9b0 con 0x7fa294069e00 2020-05-18T21:40:34.864 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa29a215700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2940a5ea0 con 0x7fa29409df60 2020-05-18T21:40:34.864 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa29a215700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2940a1620 con 0x7fa2940a1840 2020-05-18T21:40:34.865 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa299a14700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa29409df60 0x7fa294140740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43150/0 (socket says 172.21.15.196:43150) 2020-05-18T21:40:34.865 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa299a14700 1 -- 172.21.15.196:0/2274284643 learned_addr learned my addr 172.21.15.196:0/2274284643 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:34.865 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa298a12700 1 --2- 172.21.15.196:0/2274284643 >> v2:172.21.15.196:3300/0 conn(0x7fa2940a1840 0x7fa2940a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.865 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa283fff700 1 -- 172.21.15.196:0/2274284643 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966826883 0 0) 0x7fa2940a5ea0 con 0x7fa29409df60 2020-05-18T21:40:34.866 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa283fff700 1 -- 172.21.15.196:0/2274284643 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa27c003270 con 0x7fa29409df60 2020-05-18T21:40:34.866 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa298a12700 1 -- 172.21.15.196:0/2274284643 >> v1:172.21.15.196:6789/0 conn(0x7fa29409df60 legacy=0x7fa294140740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.866 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa298a12700 1 -- 172.21.15.196:0/2274284643 >> v1:172.21.15.80:6789/0 conn(0x7fa294069e00 legacy=0x7fa29409d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.866 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa298a12700 1 -- 172.21.15.196:0/2274284643 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2940a5960 con 0x7fa2940a1840 2020-05-18T21:40:34.866 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa298a12700 1 --2- 172.21.15.196:0/2274284643 >> v2:172.21.15.196:3300/0 conn(0x7fa2940a1840 0x7fa2940a48a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fa284002f70 tx=0x7fa284007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.867 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa298a12700 1 --2- 172.21.15.196:0/2274284643 >> v2:172.21.15.196:3300/0 conn(0x7fa2940a1840 0x7fa2940a48a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fa284002f70 tx=0x7fa284007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.867 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa298a12700 1 --2- 172.21.15.196:0/2274284643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa2940a48a0 secure :-1 s=READY pgs=111 cs=0 l=1 rx=0x7fa284002f70 tx=0x7fa284007a80).ready entity=mon.1 client_cookie=58fa3ae5c47a1a5a server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:34.867 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa298a12700 1 --2- 172.21.15.196:0/2274284643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa2940a48a0 secure :-1 s=THROTTLE_DONE pgs=111 cs=0 l=1 rx=0x7fa284002f70 tx=0x7fa284007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.867 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa298a12700 1 --2- 172.21.15.196:0/2274284643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa2940a48a0 secure :-1 s=THROTTLE_DONE pgs=111 cs=0 l=1 rx=0x7fa284002f70 tx=0x7fa284007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.867 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa283fff700 1 -- 172.21.15.196:0/2274284643 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa284008e80 con 0x7fa2940a1840 2020-05-18T21:40:34.868 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa298a12700 1 --2- 172.21.15.196:0/2274284643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa2940a48a0 secure :-1 s=THROTTLE_DONE pgs=111 cs=0 l=1 rx=0x7fa284002f70 tx=0x7fa284007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.868 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa283fff700 1 -- 172.21.15.196:0/2274284643 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2840115f0 con 0x7fa2940a1840 2020-05-18T21:40:34.868 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa283fff700 1 -- 172.21.15.196:0/2274284643 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2840119d0 con 0x7fa2940a1840 2020-05-18T21:40:34.868 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa29a215700 1 -- 172.21.15.196:0/2274284643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 msgr2=0x7fa2940a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.868 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa29a215700 1 --2- 172.21.15.196:0/2274284643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa2940a48a0 secure :-1 s=READY pgs=111 cs=0 l=1 rx=0x7fa284002f70 tx=0x7fa284007a80).stop 2020-05-18T21:40:34.872 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa29a215700 1 -- 172.21.15.196:0/2274284643 shutdown_connections 2020-05-18T21:40:34.872 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa29a215700 1 --2- 172.21.15.196:0/2274284643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa2940a48a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.872 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa29a215700 1 -- 172.21.15.196:0/2274284643 >> 172.21.15.196:0/2274284643 conn(0x7fa29405b290 msgr2=0x7fa2940aad10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:34.873 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa29a215700 1 -- 172.21.15.196:0/2274284643 shutdown_connections 2020-05-18T21:40:34.873 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa29a215700 1 -- 172.21.15.196:0/2274284643 wait complete. 2020-05-18T21:40:34.873 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.863+0000 7fa29a215700 1 Processor -- start 2020-05-18T21:40:34.873 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa29a215700 1 -- start start 2020-05-18T21:40:34.873 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa29a215700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa294069e00 0x7fa294135ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:34.874 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa29a215700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa29409df60 0x7fa294146f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:34.874 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa29a215700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa2940a1840 0x7fa294139a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:34.874 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa29a215700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa29413a2c0 con 0x7fa29409df60 2020-05-18T21:40:34.874 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa29a215700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa294146890 con 0x7fa2940a1840 2020-05-18T21:40:34.874 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa29a215700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa294146af0 con 0x7fa294069e00 2020-05-18T21:40:34.875 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa2940a1840 0x7fa294139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.875 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa2940a1840 0x7fa294139a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34392/0 (socket says 172.21.15.196:34392) 2020-05-18T21:40:34.875 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 -- 172.21.15.196:0/2328361472 learned_addr learned my addr 172.21.15.196:0/2328361472 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:34.875 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa298a12700 1 --2- 172.21.15.196:0/2328361472 >> v2:172.21.15.80:3300/0 conn(0x7fa29409df60 0x7fa294146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.875 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299213700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa294069e00 0x7fa294135ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.876 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 -- 172.21.15.196:0/2328361472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa294069e00 msgr2=0x7fa294135ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.876 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa294069e00 0x7fa294135ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.876 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 -- 172.21.15.196:0/2328361472 >> v2:172.21.15.80:3300/0 conn(0x7fa29409df60 msgr2=0x7fa294146f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.876 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 --2- 172.21.15.196:0/2328361472 >> v2:172.21.15.80:3300/0 conn(0x7fa29409df60 0x7fa294146f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.876 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 -- 172.21.15.196:0/2328361472 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa294136fa0 con 0x7fa2940a1840 2020-05-18T21:40:34.876 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa298a12700 1 --2- 172.21.15.196:0/2328361472 >> v2:172.21.15.80:3300/0 conn(0x7fa29409df60 0x7fa294146f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request state changed! 2020-05-18T21:40:34.877 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299213700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa294069e00 0x7fa294135ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request state changed! 2020-05-18T21:40:34.877 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 --2- 172.21.15.196:0/2328361472 >> v2:172.21.15.196:3300/0 conn(0x7fa2940a1840 0x7fa294139a20 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa28c001490 tx=0x7fa28c010d60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.877 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 --2- 172.21.15.196:0/2328361472 >> v2:172.21.15.196:3300/0 conn(0x7fa2940a1840 0x7fa294139a20 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa28c001490 tx=0x7fa28c010d60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.877 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa294139a20 secure :-1 s=READY pgs=112 cs=0 l=1 rx=0x7fa28c001490 tx=0x7fa28c010d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:34.877 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa294139a20 secure :-1 s=THROTTLE_DONE pgs=112 cs=0 l=1 rx=0x7fa28c001490 tx=0x7fa28c010d60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.878 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa2827fc700 1 -- 172.21.15.196:0/2328361472 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa28c005e90 con 0x7fa2940a1840 2020-05-18T21:40:34.878 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa294139a20 secure :-1 s=THROTTLE_DONE pgs=112 cs=0 l=1 rx=0x7fa28c001490 tx=0x7fa28c010d60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.878 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa294139a20 secure :-1 s=THROTTLE_DONE pgs=112 cs=0 l=1 rx=0x7fa28c001490 tx=0x7fa28c010d60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.878 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa2827fc700 1 -- 172.21.15.196:0/2328361472 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28c0125f0 con 0x7fa2940a1840 2020-05-18T21:40:34.879 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa29a215700 1 -- 172.21.15.196:0/2328361472 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa294137190 con 0x7fa2940a1840 2020-05-18T21:40:34.879 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa2827fc700 1 -- 172.21.15.196:0/2328361472 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa28c012a10 con 0x7fa2940a1840 2020-05-18T21:40:34.879 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa29a215700 1 -- 172.21.15.196:0/2328361472 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa294062550 con 0x7fa2940a1840 2020-05-18T21:40:34.879 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa294139a20 secure :-1 s=THROTTLE_DONE pgs=112 cs=0 l=1 rx=0x7fa28c001490 tx=0x7fa28c010d60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.879 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa29a215700 1 -- 172.21.15.196:0/2328361472 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2940569e0 con 0x7fa2940a1840 2020-05-18T21:40:34.880 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa2827fc700 1 -- 172.21.15.196:0/2328361472 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fa28c012c80 con 0x7fa2940a1840 2020-05-18T21:40:34.880 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.867+0000 7fa299a14700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa294139a20 secure :-1 s=THROTTLE_DONE pgs=112 cs=0 l=1 rx=0x7fa28c001490 tx=0x7fa28c010d60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.880 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.871+0000 7fa2827fc700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa27404f990 0x7fa274051e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:34.880 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.871+0000 7fa2827fc700 1 -- 172.21.15.196:0/2328361472 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa28c0090c0 con 0x7fa2940a1840 2020-05-18T21:40:34.880 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.871+0000 7fa299213700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa27404f990 0x7fa274051e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:34.880 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.871+0000 7fa299213700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa27404f990 0x7fa274051e50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa2940607a0 tx=0x7fa290001680).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.881 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.871+0000 7fa299213700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa27404f990 0x7fa274051e50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa2940607a0 tx=0x7fa290001680).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.881 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.871+0000 7fa299213700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa27404f990 0x7fa274051e50 secure :-1 s=READY pgs=41 cs=0 l=1 rx=0x7fa2940607a0 tx=0x7fa290001680).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:34.882 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.875+0000 7fa299a14700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa294139a20 secure :-1 s=THROTTLE_DONE pgs=112 cs=0 l=1 rx=0x7fa28c001490 tx=0x7fa28c010d60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.882 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.875+0000 7fa2827fc700 1 -- 172.21.15.196:0/2328361472 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7fa28c03e4f0 con 0x7fa2940a1840 2020-05-18T21:40:34.993 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa29a215700 1 -- 172.21.15.196:0/2328361472 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa2940abb90 con 0x7fa2940a1840 2020-05-18T21:40:34.994 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa299a14700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa294139a20 secure :-1 s=THROTTLE_DONE pgs=112 cs=0 l=1 rx=0x7fa28c001490 tx=0x7fa28c010d60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:34.994 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa2827fc700 1 -- 172.21.15.196:0/2328361472 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa28c041b10 con 0x7fa2940a1840 2020-05-18T21:40:34.995 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2020-05-18T21:40:34.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa29a215700 1 -- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa27404f990 msgr2=0x7fa274051e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa29a215700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa27404f990 0x7fa274051e50 secure :-1 s=READY pgs=41 cs=0 l=1 rx=0x7fa2940607a0 tx=0x7fa290001680).stop 2020-05-18T21:40:34.996 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa29a215700 1 -- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 msgr2=0x7fa294139a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:34.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa29a215700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa294139a20 secure :-1 s=READY pgs=112 cs=0 l=1 rx=0x7fa28c001490 tx=0x7fa28c010d60).stop 2020-05-18T21:40:34.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa29a215700 1 -- 172.21.15.196:0/2328361472 shutdown_connections 2020-05-18T21:40:34.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa29a215700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2940a1840 0x7fa294139a20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa29a215700 1 --2- 172.21.15.196:0/2328361472 >> v2:172.21.15.80:3300/0 conn(0x7fa29409df60 0x7fa294146f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.997 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa29a215700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa27404f990 0x7fa274051e50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa29a215700 1 --2- 172.21.15.196:0/2328361472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa294069e00 0x7fa294135ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:34.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa29a215700 1 -- 172.21.15.196:0/2328361472 >> 172.21.15.196:0/2328361472 conn(0x7fa29405b290 msgr2=0x7fa29405eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:34.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa29a215700 1 -- 172.21.15.196:0/2328361472 shutdown_connections 2020-05-18T21:40:34.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:34.995+0000 7fa29a215700 1 -- 172.21.15.196:0/2328361472 wait complete. 2020-05-18T21:40:35.007 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-05-18T21:40:35.114 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9d0cfc700 1 Processor -- start 2020-05-18T21:40:35.115 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9d0cfc700 1 -- start start 2020-05-18T21:40:35.117 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9d0cfc700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9cc13fd80 0x7fc9cc140210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:35.117 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9d0cfc700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc9cc140670 0x7fc9cc1471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:35.117 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9d0cfc700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9cc0a7750 con 0x7fc9cc05b210 2020-05-18T21:40:35.118 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9d0cfc700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9cc0a4e10 con 0x7fc9cc140670 2020-05-18T21:40:35.118 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9d0cfc700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0a7980 con 0x7fc9cc13fd80 2020-05-18T21:40:35.118 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9cb7fe700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc9cc05b210 0x7fc9cc0a33a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43160/0 (socket says 172.21.15.196:43160) 2020-05-18T21:40:35.118 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9cb7fe700 1 -- 172.21.15.196:0/2667118536 learned_addr learned my addr 172.21.15.196:0/2667118536 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:35.119 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9cbfff700 1 --2- 172.21.15.196:0/2667118536 >> v2:172.21.15.80:3300/0 conn(0x7fc9cc140670 0x7fc9cc1471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:35.119 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9caffd700 1 --2- 172.21.15.196:0/2667118536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9cc13fd80 0x7fc9cc140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:35.119 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9ca7fc700 1 -- 172.21.15.196:0/2667118536 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040611015 0 0) 0x7fc9cc0a7750 con 0x7fc9cc05b210 2020-05-18T21:40:35.119 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9ca7fc700 1 -- 172.21.15.196:0/2667118536 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9ac003270 con 0x7fc9cc05b210 2020-05-18T21:40:35.119 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9ca7fc700 1 -- 172.21.15.196:0/2667118536 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1445936940 0 0) 0x7fc9ac003270 con 0x7fc9cc05b210 2020-05-18T21:40:35.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9cbfff700 1 -- 172.21.15.196:0/2667118536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9cc13fd80 msgr2=0x7fc9cc140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:35.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9cbfff700 1 --2- 172.21.15.196:0/2667118536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9cc13fd80 0x7fc9cc140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:35.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9cbfff700 1 -- 172.21.15.196:0/2667118536 >> v1:172.21.15.196:6789/0 conn(0x7fc9cc05b210 legacy=0x7fc9cc0a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9cbfff700 1 -- 172.21.15.196:0/2667118536 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc0a7c20 con 0x7fc9cc140670 2020-05-18T21:40:35.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9cbfff700 1 --2- 172.21.15.196:0/2667118536 >> v2:172.21.15.80:3300/0 conn(0x7fc9cc140670 0x7fc9cc1471e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fc9bc0018b0 tx=0x7fc9c4007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9cbfff700 1 --2- 172.21.15.196:0/2667118536 >> v2:172.21.15.80:3300/0 conn(0x7fc9cc140670 0x7fc9cc1471e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fc9bc0018b0 tx=0x7fc9c4007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9cbfff700 1 --2- 172.21.15.196:0/2667118536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc140670 0x7fc9cc1471e0 secure :-1 s=READY pgs=86 cs=0 l=1 rx=0x7fc9bc0018b0 tx=0x7fc9c4007e00).ready entity=mon.0 client_cookie=f40e68936bb1819d server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:35.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9cbfff700 1 --2- 172.21.15.196:0/2667118536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc140670 0x7fc9cc1471e0 secure :-1 s=THROTTLE_DONE pgs=86 cs=0 l=1 rx=0x7fc9bc0018b0 tx=0x7fc9c4007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9ca7fc700 1 -- 172.21.15.196:0/2667118536 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9bc001710 con 0x7fc9cc140670 2020-05-18T21:40:35.122 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9cbfff700 1 --2- 172.21.15.196:0/2667118536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc140670 0x7fc9cc1471e0 secure :-1 s=THROTTLE_DONE pgs=86 cs=0 l=1 rx=0x7fc9bc0018b0 tx=0x7fc9c4007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.122 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9cbfff700 1 --2- 172.21.15.196:0/2667118536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc140670 0x7fc9cc1471e0 secure :-1 s=THROTTLE_DONE pgs=86 cs=0 l=1 rx=0x7fc9bc0018b0 tx=0x7fc9c4007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.122 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9ca7fc700 1 -- 172.21.15.196:0/2667118536 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c4011ac0 con 0x7fc9cc140670 2020-05-18T21:40:35.122 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9ca7fc700 1 -- 172.21.15.196:0/2667118536 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9c4011e20 con 0x7fc9cc140670 2020-05-18T21:40:35.122 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2667118536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc140670 msgr2=0x7fc9cc1471e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.123 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.115+0000 7fc9d0cfc700 1 --2- 172.21.15.196:0/2667118536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc140670 0x7fc9cc1471e0 secure :-1 s=READY pgs=86 cs=0 l=1 rx=0x7fc9bc0018b0 tx=0x7fc9c4007e00).stop 2020-05-18T21:40:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2667118536 shutdown_connections 2020-05-18T21:40:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9d0cfc700 1 --2- 172.21.15.196:0/2667118536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc140670 0x7fc9cc1471e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9d0cfc700 1 --2- 172.21.15.196:0/2667118536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9cc13fd80 0x7fc9cc140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2667118536 >> 172.21.15.196:0/2667118536 conn(0x7fc9cc060640 msgr2=0x7fc9cc05a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2667118536 shutdown_connections 2020-05-18T21:40:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2667118536 wait complete. 2020-05-18T21:40:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9d0cfc700 1 Processor -- start 2020-05-18T21:40:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9d0cfc700 1 -- start start 2020-05-18T21:40:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9d0cfc700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc9cc05b210 0x7fc9cc05ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9d0cfc700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9cc149620 con 0x7fc9cc13fd80 2020-05-18T21:40:35.129 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9d0cfc700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9cc1498c0 con 0x7fc9cc140670 2020-05-18T21:40:35.129 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9d0cfc700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9cc149b60 con 0x7fc9cc05b210 2020-05-18T21:40:35.129 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9cbfff700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc9cc140670 0x7fc9cc153e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43168/0 (socket says 172.21.15.196:43168) 2020-05-18T21:40:35.129 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9cbfff700 1 -- 172.21.15.196:0/2598400939 learned_addr learned my addr 172.21.15.196:0/2598400939 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:35.130 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9cb7fe700 1 --2- 172.21.15.196:0/2598400939 >> v2:172.21.15.80:3300/0 conn(0x7fc9cc05b210 0x7fc9cc05ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:35.130 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.119+0000 7fc9c8ff9700 1 -- 172.21.15.196:0/2598400939 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824112868 0 0) 0x7fc9cc1498c0 con 0x7fc9cc140670 2020-05-18T21:40:35.130 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9c8ff9700 1 -- 172.21.15.196:0/2598400939 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9b0002fe0 con 0x7fc9cc140670 2020-05-18T21:40:35.130 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9cb7fe700 1 -- 172.21.15.196:0/2598400939 >> v1:172.21.15.196:6789/0 conn(0x7fc9cc140670 legacy=0x7fc9cc153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.130 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9cb7fe700 1 -- 172.21.15.196:0/2598400939 >> v1:172.21.15.80:6789/0 conn(0x7fc9cc13fd80 legacy=0x7fc9cc0a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.131 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9cb7fe700 1 -- 172.21.15.196:0/2598400939 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc0a8e80 con 0x7fc9cc05b210 2020-05-18T21:40:35.131 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9cb7fe700 1 --2- 172.21.15.196:0/2598400939 >> v2:172.21.15.80:3300/0 conn(0x7fc9cc05b210 0x7fc9cc05ee90 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc9bc001970 tx=0x7fc9bc010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.131 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9cb7fe700 1 --2- 172.21.15.196:0/2598400939 >> v2:172.21.15.80:3300/0 conn(0x7fc9cc05b210 0x7fc9cc05ee90 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc9bc001970 tx=0x7fc9bc010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.131 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9cb7fe700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc05b210 0x7fc9cc05ee90 secure :-1 s=READY pgs=88 cs=0 l=1 rx=0x7fc9bc001970 tx=0x7fc9bc010c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:35.131 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9cb7fe700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc05b210 0x7fc9cc05ee90 secure :-1 s=THROTTLE_DONE pgs=88 cs=0 l=1 rx=0x7fc9bc001970 tx=0x7fc9bc010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.132 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9c8ff9700 1 -- 172.21.15.196:0/2598400939 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9bc0129b0 con 0x7fc9cc05b210 2020-05-18T21:40:35.132 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9cb7fe700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc05b210 0x7fc9cc05ee90 secure :-1 s=THROTTLE_DONE pgs=88 cs=0 l=1 rx=0x7fc9bc001970 tx=0x7fc9bc010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.132 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9cb7fe700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc05b210 0x7fc9cc05ee90 secure :-1 s=THROTTLE_DONE pgs=88 cs=0 l=1 rx=0x7fc9bc001970 tx=0x7fc9bc010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.132 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9c8ff9700 1 -- 172.21.15.196:0/2598400939 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9bc012b10 con 0x7fc9cc05b210 2020-05-18T21:40:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9c8ff9700 1 -- 172.21.15.196:0/2598400939 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9bc012e70 con 0x7fc9cc05b210 2020-05-18T21:40:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2598400939 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9cc0a1070 con 0x7fc9cc05b210 2020-05-18T21:40:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2598400939 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9cc09fc50 con 0x7fc9cc05b210 2020-05-18T21:40:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2598400939 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9cc0569e0 con 0x7fc9cc05b210 2020-05-18T21:40:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9cb7fe700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc05b210 0x7fc9cc05ee90 secure :-1 s=THROTTLE_DONE pgs=88 cs=0 l=1 rx=0x7fc9bc001970 tx=0x7fc9bc010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9c8ff9700 1 -- 172.21.15.196:0/2598400939 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fc9bc0101e0 con 0x7fc9cc05b210 2020-05-18T21:40:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9cb7fe700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc05b210 0x7fc9cc05ee90 secure :-1 s=THROTTLE_DONE pgs=88 cs=0 l=1 rx=0x7fc9bc001970 tx=0x7fc9bc010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9c8ff9700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc9b004fcf0 0x7fc9b00521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9c8ff9700 1 -- 172.21.15.196:0/2598400939 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc9bc063e40 con 0x7fc9cc05b210 2020-05-18T21:40:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.123+0000 7fc9caffd700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc9b004fcf0 0x7fc9b00521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.127+0000 7fc9caffd700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc9b004fcf0 0x7fc9b00521b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc9cc13eb20 tx=0x7fc9b4008780).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.127+0000 7fc9caffd700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc9b004fcf0 0x7fc9b00521b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc9cc13eb20 tx=0x7fc9b4008780).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.127+0000 7fc9caffd700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc9b004fcf0 0x7fc9b00521b0 secure :-1 s=READY pgs=42 cs=0 l=1 rx=0x7fc9cc13eb20 tx=0x7fc9b4008780).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:35.136 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.135+0000 7fc9cb7fe700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc05b210 0x7fc9cc05ee90 secure :-1 s=THROTTLE_DONE pgs=88 cs=0 l=1 rx=0x7fc9bc001970 tx=0x7fc9bc010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.136 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.135+0000 7fc9c8ff9700 1 -- 172.21.15.196:0/2598400939 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7fc9bc03e1d0 con 0x7fc9cc05b210 2020-05-18T21:40:35.258 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.259+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2598400939 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc9cc0a1200 con 0x7fc9cc05b210 2020-05-18T21:40:35.259 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.259+0000 7fc9cb7fe700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc05b210 0x7fc9cc05ee90 secure :-1 s=THROTTLE_DONE pgs=88 cs=0 l=1 rx=0x7fc9bc001970 tx=0x7fc9bc010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.259 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.259+0000 7fc9c8ff9700 1 -- 172.21.15.196:0/2598400939 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc9bc0417f0 con 0x7fc9cc05b210 2020-05-18T21:40:35.260 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 8 2020-05-18T21:40:35.260 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.259+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2598400939 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc9b004fcf0 msgr2=0x7fc9b00521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.261 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.259+0000 7fc9d0cfc700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc9b004fcf0 0x7fc9b00521b0 secure :-1 s=READY pgs=42 cs=0 l=1 rx=0x7fc9cc13eb20 tx=0x7fc9b4008780).stop 2020-05-18T21:40:35.261 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.259+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2598400939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc05b210 msgr2=0x7fc9cc05ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.261 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.259+0000 7fc9d0cfc700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc05b210 0x7fc9cc05ee90 secure :-1 s=READY pgs=88 cs=0 l=1 rx=0x7fc9bc001970 tx=0x7fc9bc010c30).stop 2020-05-18T21:40:35.262 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.259+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2598400939 shutdown_connections 2020-05-18T21:40:35.262 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.259+0000 7fc9d0cfc700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc9b004fcf0 0x7fc9b00521b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:35.262 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.259+0000 7fc9d0cfc700 1 --2- 172.21.15.196:0/2598400939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9cc05b210 0x7fc9cc05ee90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:35.262 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.259+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2598400939 >> 172.21.15.196:0/2598400939 conn(0x7fc9cc060640 msgr2=0x7fc9cc0a17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:35.262 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.259+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2598400939 shutdown_connections 2020-05-18T21:40:35.263 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.259+0000 7fc9d0cfc700 1 -- 172.21.15.196:0/2598400939 wait complete. 2020-05-18T21:40:35.270 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-05-18T21:40:35.367 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.367+0000 7f8690b1f700 1 Processor -- start 2020-05-18T21:40:35.368 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.367+0000 7f8690b1f700 1 -- start start 2020-05-18T21:40:35.370 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.367+0000 7f8690b1f700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f868c0ab7c0 0x7f868c13bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:35.371 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.367+0000 7f8690b1f700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868c066a50 con 0x7f868c13c130 2020-05-18T21:40:35.371 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.367+0000 7f8690b1f700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868c0aa9a0 con 0x7f868c0a8720 2020-05-18T21:40:35.371 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.367+0000 7f8690b1f700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868c0a94a0 con 0x7f868c0ab7c0 2020-05-18T21:40:35.371 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.367+0000 7f868a59c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f868c0ab7c0 0x7f868c13bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:35.372 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.367+0000 7f8681d9b700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f868c0a8720 0x7f868c065c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43174/0 (socket says 172.21.15.196:43174) 2020-05-18T21:40:35.372 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.367+0000 7f8681d9b700 1 -- 172.21.15.196:0/112226760 learned_addr learned my addr 172.21.15.196:0/112226760 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:35.372 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.367+0000 7f8689d9b700 1 -- 172.21.15.196:0/112226760 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298711152 0 0) 0x7f868c0aa9a0 con 0x7f868c0a8720 2020-05-18T21:40:35.372 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f8689d9b700 1 -- 172.21.15.196:0/112226760 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f866c003270 con 0x7f868c0a8720 2020-05-18T21:40:35.372 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f868a59c700 1 -- 172.21.15.196:0/112226760 >> v1:172.21.15.196:6789/0 conn(0x7f868c0a8720 legacy=0x7f868c065c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.373 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f868a59c700 1 -- 172.21.15.196:0/112226760 >> v1:172.21.15.80:6789/0 conn(0x7f868c13c130 legacy=0x7f868c066340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.373 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f868a59c700 1 -- 172.21.15.196:0/112226760 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c0a9740 con 0x7f868c0ab7c0 2020-05-18T21:40:35.373 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f868a59c700 1 --2- 172.21.15.196:0/112226760 >> v2:172.21.15.196:3300/0 conn(0x7f868c0ab7c0 0x7f868c13bcd0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.373 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f868a59c700 1 --2- 172.21.15.196:0/112226760 >> v2:172.21.15.196:3300/0 conn(0x7f868c0ab7c0 0x7f868c13bcd0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.373 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f868a59c700 1 --2- 172.21.15.196:0/112226760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0ab7c0 0x7f868c13bcd0 secure :-1 s=READY pgs=116 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).ready entity=mon.1 client_cookie=902fd2de26bced9d server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:35.374 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f868a59c700 1 --2- 172.21.15.196:0/112226760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0ab7c0 0x7f868c13bcd0 secure :-1 s=THROTTLE_DONE pgs=116 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.374 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f868a59c700 1 --2- 172.21.15.196:0/112226760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0ab7c0 0x7f868c13bcd0 secure :-1 s=THROTTLE_DONE pgs=116 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.374 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f8689d9b700 1 -- 172.21.15.196:0/112226760 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f867c008e40 con 0x7f868c0ab7c0 2020-05-18T21:40:35.374 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f868a59c700 1 --2- 172.21.15.196:0/112226760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0ab7c0 0x7f868c13bcd0 secure :-1 s=THROTTLE_DONE pgs=116 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.375 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f8689d9b700 1 -- 172.21.15.196:0/112226760 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c0114f0 con 0x7f868c0ab7c0 2020-05-18T21:40:35.375 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f8689d9b700 1 -- 172.21.15.196:0/112226760 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f867c0118d0 con 0x7f868c0ab7c0 2020-05-18T21:40:35.375 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f8690b1f700 1 -- 172.21.15.196:0/112226760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0ab7c0 msgr2=0x7f868c13bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.376 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f8690b1f700 1 --2- 172.21.15.196:0/112226760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0ab7c0 0x7f868c13bcd0 secure :-1 s=READY pgs=116 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).stop 2020-05-18T21:40:35.380 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f8690b1f700 1 -- 172.21.15.196:0/112226760 shutdown_connections 2020-05-18T21:40:35.381 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f8690b1f700 1 --2- 172.21.15.196:0/112226760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0ab7c0 0x7f868c13bcd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:35.382 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f8690b1f700 1 -- 172.21.15.196:0/112226760 >> 172.21.15.196:0/112226760 conn(0x7f868c069ac0 msgr2=0x7f868c069ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:35.382 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f8690b1f700 1 -- 172.21.15.196:0/112226760 shutdown_connections 2020-05-18T21:40:35.382 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.371+0000 7f8690b1f700 1 -- 172.21.15.196:0/112226760 wait complete. 2020-05-18T21:40:35.382 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8690b1f700 1 Processor -- start 2020-05-18T21:40:35.382 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8690b1f700 1 -- start start 2020-05-18T21:40:35.383 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8690b1f700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f868c0a8720 0x7f868c09fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:35.383 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8690b1f700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f868c0ab7c0 0x7f868c1491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:35.383 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8690b1f700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868c146660 con 0x7f868c13c130 2020-05-18T21:40:35.383 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8690b1f700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868c0ae5a0 con 0x7f868c0ab7c0 2020-05-18T21:40:35.384 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8690b1f700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868c149af0 con 0x7f868c0a8720 2020-05-18T21:40:35.384 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868a59c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f868c0a8720 0x7f868c09fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:35.384 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868a59c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f868c0a8720 0x7f868c09fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34420/0 (socket says 172.21.15.196:34420) 2020-05-18T21:40:35.384 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868a59c700 1 -- 172.21.15.196:0/1369007992 learned_addr learned my addr 172.21.15.196:0/1369007992 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:35.384 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868ad9d700 1 --1- 172.21.15.196:0/1369007992 >> v1:172.21.15.196:6789/0 conn(0x7f868c13c130 0x7f868c145f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43184/0 (socket says 172.21.15.196:43184) 2020-05-18T21:40:35.385 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8681d9b700 1 --2- 172.21.15.196:0/1369007992 >> v2:172.21.15.80:3300/0 conn(0x7f868c0ab7c0 0x7f868c1491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:35.385 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8683fff700 1 -- 172.21.15.196:0/1369007992 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339828401 0 0) 0x7f868c146660 con 0x7f868c13c130 2020-05-18T21:40:35.385 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8683fff700 1 -- 172.21.15.196:0/1369007992 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8670002fe0 con 0x7f868c13c130 2020-05-18T21:40:35.385 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868a59c700 1 -- 172.21.15.196:0/1369007992 >> v2:172.21.15.80:3300/0 conn(0x7f868c0ab7c0 msgr2=0x7f868c1491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868a59c700 1 --2- 172.21.15.196:0/1369007992 >> v2:172.21.15.80:3300/0 conn(0x7f868c0ab7c0 0x7f868c1491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868a59c700 1 -- 172.21.15.196:0/1369007992 >> v1:172.21.15.196:6789/0 conn(0x7f868c13c130 legacy=0x7f868c145f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868a59c700 1 -- 172.21.15.196:0/1369007992 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c145810 con 0x7f868c0a8720 2020-05-18T21:40:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868a59c700 1 --2- 172.21.15.196:0/1369007992 >> v2:172.21.15.196:3300/0 conn(0x7f868c0a8720 0x7f868c09fe30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.386 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868a59c700 1 --2- 172.21.15.196:0/1369007992 >> v2:172.21.15.196:3300/0 conn(0x7f868c0a8720 0x7f868c09fe30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.387 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868a59c700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0a8720 0x7f868c09fe30 secure :-1 s=READY pgs=118 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:35.387 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868a59c700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0a8720 0x7f868c09fe30 secure :-1 s=THROTTLE_DONE pgs=118 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.387 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8683fff700 1 -- 172.21.15.196:0/1369007992 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f867c011800 con 0x7f868c0a8720 2020-05-18T21:40:35.387 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868a59c700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0a8720 0x7f868c09fe30 secure :-1 s=THROTTLE_DONE pgs=118 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.388 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8683fff700 1 -- 172.21.15.196:0/1369007992 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c011960 con 0x7f868c0a8720 2020-05-18T21:40:35.388 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f868a59c700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0a8720 0x7f868c09fe30 secure :-1 s=THROTTLE_DONE pgs=118 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.388 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8690b1f700 1 -- 172.21.15.196:0/1369007992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c145ab0 con 0x7f868c0a8720 2020-05-18T21:40:35.388 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.375+0000 7f8683fff700 1 -- 172.21.15.196:0/1369007992 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f867c00c5a0 con 0x7f868c0a8720 2020-05-18T21:40:35.388 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.379+0000 7f8690b1f700 1 -- 172.21.15.196:0/1369007992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c149600 con 0x7f868c0a8720 2020-05-18T21:40:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.379+0000 7f8690b1f700 1 -- 172.21.15.196:0/1369007992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868c149910 con 0x7f868c0a8720 2020-05-18T21:40:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.379+0000 7f868a59c700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0a8720 0x7f868c09fe30 secure :-1 s=THROTTLE_DONE pgs=118 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.379+0000 7f8683fff700 1 -- 172.21.15.196:0/1369007992 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f867c00c7a0 con 0x7f868c0a8720 2020-05-18T21:40:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.379+0000 7f868a59c700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0a8720 0x7f868c09fe30 secure :-1 s=THROTTLE_DONE pgs=118 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.389 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.379+0000 7f8683fff700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f867004fd80 0x7f8670052240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:35.390 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.379+0000 7f8681d9b700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f867004fd80 0x7f8670052240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:35.390 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.379+0000 7f8683fff700 1 -- 172.21.15.196:0/1369007992 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f867c064a80 con 0x7f868c0a8720 2020-05-18T21:40:35.390 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.379+0000 7f8681d9b700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f867004fd80 0x7f8670052240 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8674001110 tx=0x7f867400c950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.390 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.379+0000 7f8681d9b700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f867004fd80 0x7f8670052240 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8674001110 tx=0x7f867400c950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.391 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.379+0000 7f8681d9b700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f867004fd80 0x7f8670052240 secure :-1 s=READY pgs=44 cs=0 l=1 rx=0x7f8674001110 tx=0x7f867400c950).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:35.391 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.383+0000 7f868a59c700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0a8720 0x7f868c09fe30 secure :-1 s=THROTTLE_DONE pgs=118 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.392 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.383+0000 7f8683fff700 1 -- 172.21.15.196:0/1369007992 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f867c03edb0 con 0x7f868c0a8720 2020-05-18T21:40:35.502 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f8690b1f700 1 -- 172.21.15.196:0/1369007992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f868c0569e0 con 0x7f868c0a8720 2020-05-18T21:40:35.503 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f868a59c700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0a8720 0x7f868c09fe30 secure :-1 s=THROTTLE_DONE pgs=118 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.503 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f8683fff700 1 -- 172.21.15.196:0/1369007992 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f867c0423d0 con 0x7f868c0a8720 2020-05-18T21:40:35.503 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 8 2020-05-18T21:40:35.504 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f8690b1f700 1 -- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f867004fd80 msgr2=0x7f8670052240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.504 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f8690b1f700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f867004fd80 0x7f8670052240 secure :-1 s=READY pgs=44 cs=0 l=1 rx=0x7f8674001110 tx=0x7f867400c950).stop 2020-05-18T21:40:35.505 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f8690b1f700 1 -- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0a8720 msgr2=0x7f868c09fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.505 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f8690b1f700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0a8720 0x7f868c09fe30 secure :-1 s=READY pgs=118 cs=0 l=1 rx=0x7f8684001490 tx=0x7f867c007a80).stop 2020-05-18T21:40:35.506 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f8690b1f700 1 -- 172.21.15.196:0/1369007992 shutdown_connections 2020-05-18T21:40:35.506 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f8690b1f700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f867004fd80 0x7f8670052240 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:35.506 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f8690b1f700 1 --2- 172.21.15.196:0/1369007992 >> v2:172.21.15.80:3300/0 conn(0x7f868c0ab7c0 0x7f868c1491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:35.507 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f8690b1f700 1 --2- 172.21.15.196:0/1369007992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c0a8720 0x7f868c09fe30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:35.507 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f8690b1f700 1 -- 172.21.15.196:0/1369007992 >> 172.21.15.196:0/1369007992 conn(0x7f868c069ac0 msgr2=0x7f868c0aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:35.507 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f8690b1f700 1 -- 172.21.15.196:0/1369007992 shutdown_connections 2020-05-18T21:40:35.507 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.503+0000 7f8690b1f700 1 -- 172.21.15.196:0/1369007992 wait complete. 2020-05-18T21:40:35.516 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-05-18T21:40:35.624 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.623+0000 7f7482d53700 1 Processor -- start 2020-05-18T21:40:35.624 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.623+0000 7f7482d53700 1 -- start start 2020-05-18T21:40:35.625 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.623+0000 7f7482d53700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f747c09ecb0 0x7f747c13dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:35.626 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.623+0000 7f7482d53700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f747c13e1a0 0x7f747c140590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:35.626 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.623+0000 7f7482d53700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f747c0a6050 con 0x7f747c05b210 2020-05-18T21:40:35.626 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.623+0000 7f7482d53700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747c0a6cc0 con 0x7f747c09ecb0 2020-05-18T21:40:35.626 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.623+0000 7f7482d53700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747c09e5e0 con 0x7f747c13e1a0 2020-05-18T21:40:35.627 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482552700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f747c13e1a0 0x7f747c140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:35.627 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7481d51700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f747c05b210 0x7f747c0a51d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43188/0 (socket says 172.21.15.196:43188) 2020-05-18T21:40:35.627 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482552700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f747c13e1a0 0x7f747c140590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34432/0 (socket says 172.21.15.196:34432) 2020-05-18T21:40:35.628 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7481d51700 1 -- 172.21.15.196:0/1164594500 learned_addr learned my addr 172.21.15.196:0/1164594500 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:35.628 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7481550700 1 --2- 172.21.15.196:0/1164594500 >> v2:172.21.15.80:3300/0 conn(0x7f747c09ecb0 0x7f747c13dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:35.630 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482552700 1 -- 172.21.15.196:0/1164594500 >> v2:172.21.15.80:3300/0 conn(0x7f747c09ecb0 msgr2=0x7f747c13dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:35.630 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482552700 1 --2- 172.21.15.196:0/1164594500 >> v2:172.21.15.80:3300/0 conn(0x7f747c09ecb0 0x7f747c13dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:35.630 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482552700 1 -- 172.21.15.196:0/1164594500 >> v1:172.21.15.196:6789/0 conn(0x7f747c05b210 legacy=0x7f747c0a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.631 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7480d4f700 1 -- 172.21.15.196:0/1164594500 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058604190 0 0) 0x7f747c0a6050 con 0x7f747c05b210 2020-05-18T21:40:35.631 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482552700 1 -- 172.21.15.196:0/1164594500 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c0a4da0 con 0x7f747c13e1a0 2020-05-18T21:40:35.631 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482552700 1 --2- 172.21.15.196:0/1164594500 >> v2:172.21.15.196:3300/0 conn(0x7f747c13e1a0 0x7f747c140590 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.632 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482552700 1 --2- 172.21.15.196:0/1164594500 >> v2:172.21.15.196:3300/0 conn(0x7f747c13e1a0 0x7f747c140590 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.632 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482552700 1 --2- 172.21.15.196:0/1164594500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c140590 secure :-1 s=READY pgs=120 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).ready entity=mon.1 client_cookie=be7619ca7d58e497 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:35.632 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482552700 1 --2- 172.21.15.196:0/1164594500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c140590 secure :-1 s=THROTTLE_DONE pgs=120 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.632 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7480d4f700 1 -- 172.21.15.196:0/1164594500 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7474011e60 con 0x7f747c13e1a0 2020-05-18T21:40:35.632 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482552700 1 --2- 172.21.15.196:0/1164594500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c140590 secure :-1 s=THROTTLE_DONE pgs=120 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.633 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7480d4f700 1 -- 172.21.15.196:0/1164594500 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747400a180 con 0x7f747c13e1a0 2020-05-18T21:40:35.633 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482552700 1 --2- 172.21.15.196:0/1164594500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c140590 secure :-1 s=THROTTLE_DONE pgs=120 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.633 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482d53700 1 -- 172.21.15.196:0/1164594500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 msgr2=0x7f747c140590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.633 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7480d4f700 1 -- 172.21.15.196:0/1164594500 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f747400a520 con 0x7f747c13e1a0 2020-05-18T21:40:35.634 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482d53700 1 --2- 172.21.15.196:0/1164594500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c140590 secure :-1 s=READY pgs=120 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).stop 2020-05-18T21:40:35.634 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482d53700 1 -- 172.21.15.196:0/1164594500 shutdown_connections 2020-05-18T21:40:35.634 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482d53700 1 --2- 172.21.15.196:0/1164594500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c140590 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:35.634 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482d53700 1 --2- 172.21.15.196:0/1164594500 >> v2:172.21.15.80:3300/0 conn(0x7f747c09ecb0 0x7f747c13dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:35.634 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482d53700 1 -- 172.21.15.196:0/1164594500 >> 172.21.15.196:0/1164594500 conn(0x7f747c060640 msgr2=0x7f747c05a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:35.635 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482d53700 1 -- 172.21.15.196:0/1164594500 shutdown_connections 2020-05-18T21:40:35.635 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.627+0000 7f7482d53700 1 -- 172.21.15.196:0/1164594500 wait complete. 2020-05-18T21:40:35.635 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482d53700 1 Processor -- start 2020-05-18T21:40:35.635 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482d53700 1 -- start start 2020-05-18T21:40:35.639 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482d53700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f747c05b210 0x7f747c0aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:35.639 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482d53700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f747c13e1a0 0x7f747c1340b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:35.639 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482d53700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f747c0a2f60 con 0x7f747c09ecb0 2020-05-18T21:40:35.639 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482d53700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747c09a3e0 con 0x7f747c13e1a0 2020-05-18T21:40:35.640 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482d53700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747c065380 con 0x7f747c05b210 2020-05-18T21:40:35.640 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482552700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f747c13e1a0 0x7f747c1340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:35.640 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482552700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f747c13e1a0 0x7f747c1340b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34438/0 (socket says 172.21.15.196:34438) 2020-05-18T21:40:35.640 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482552700 1 -- 172.21.15.196:0/869456078 learned_addr learned my addr 172.21.15.196:0/869456078 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:35.641 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7481d51700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f747c05b210 0x7f747c0aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:35.641 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482552700 1 -- 172.21.15.196:0/869456078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f747c05b210 msgr2=0x7f747c0aa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.641 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482552700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f747c05b210 0x7f747c0aa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:35.641 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482552700 1 -- 172.21.15.196:0/869456078 >> v1:172.21.15.80:6789/0 conn(0x7f747c09ecb0 legacy=0x7f747c066250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.641 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482552700 1 -- 172.21.15.196:0/869456078 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c0a31b0 con 0x7f747c13e1a0 2020-05-18T21:40:35.642 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482552700 1 --2- 172.21.15.196:0/869456078 >> v2:172.21.15.196:3300/0 conn(0x7f747c13e1a0 0x7f747c1340b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.642 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482552700 1 --2- 172.21.15.196:0/869456078 >> v2:172.21.15.196:3300/0 conn(0x7f747c13e1a0 0x7f747c1340b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.642 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482552700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c1340b0 secure :-1 s=READY pgs=121 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:35.642 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482552700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c1340b0 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.642 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482552700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c1340b0 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.643 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f746effd700 1 -- 172.21.15.196:0/869456078 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7474005c70 con 0x7f747c13e1a0 2020-05-18T21:40:35.643 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482552700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c1340b0 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.643 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f746effd700 1 -- 172.21.15.196:0/869456078 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7474005dd0 con 0x7f747c13e1a0 2020-05-18T21:40:35.643 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482d53700 1 -- 172.21.15.196:0/869456078 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f747c0a3450 con 0x7f747c13e1a0 2020-05-18T21:40:35.644 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f746effd700 1 -- 172.21.15.196:0/869456078 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f747400f7a0 con 0x7f747c13e1a0 2020-05-18T21:40:35.644 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.631+0000 7f7482d53700 1 -- 172.21.15.196:0/869456078 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747c0a38f0 con 0x7f747c13e1a0 2020-05-18T21:40:35.644 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.635+0000 7f7482552700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c1340b0 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.644 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.635+0000 7f7482d53700 1 -- 172.21.15.196:0/869456078 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747c0569e0 con 0x7f747c13e1a0 2020-05-18T21:40:35.644 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.635+0000 7f746effd700 1 -- 172.21.15.196:0/869456078 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f747400fa10 con 0x7f747c13e1a0 2020-05-18T21:40:35.645 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.635+0000 7f7482552700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c1340b0 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.645 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.635+0000 7f746effd700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f745c04f9f0 0x7f745c051eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:35.645 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.635+0000 7f746effd700 1 -- 172.21.15.196:0/869456078 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7474064dd0 con 0x7f747c13e1a0 2020-05-18T21:40:35.645 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.635+0000 7f7481d51700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f745c04f9f0 0x7f745c051eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:35.646 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.635+0000 7f7481d51700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f745c04f9f0 0x7f745c051eb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7470000bc0 tx=0x7f7470001750).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.646 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.635+0000 7f7481d51700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f745c04f9f0 0x7f745c051eb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7470000bc0 tx=0x7f7470001750).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.646 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.635+0000 7f7481d51700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f745c04f9f0 0x7f745c051eb0 secure :-1 s=READY pgs=45 cs=0 l=1 rx=0x7f7470000bc0 tx=0x7f7470001750).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:35.646 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.643+0000 7f7482552700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c1340b0 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.647 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.643+0000 7f746effd700 1 -- 172.21.15.196:0/869456078 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f747403f220 con 0x7f747c13e1a0 2020-05-18T21:40:35.757 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.755+0000 7f7482d53700 1 -- 172.21.15.196:0/869456078 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f747c0a35e0 con 0x7f747c13e1a0 2020-05-18T21:40:35.880 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.755+0000 7f7482552700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c1340b0 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:35.881 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.755+0000 7f746effd700 1 -- 172.21.15.196:0/869456078 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7474042840 con 0x7f747c13e1a0 2020-05-18T21:40:35.882 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 8 2020-05-18T21:40:35.885 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.759+0000 7f7482d53700 1 -- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f745c04f9f0 msgr2=0x7f745c051eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.885 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.759+0000 7f7482d53700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f745c04f9f0 0x7f745c051eb0 secure :-1 s=READY pgs=45 cs=0 l=1 rx=0x7f7470000bc0 tx=0x7f7470001750).stop 2020-05-18T21:40:35.886 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.759+0000 7f7482d53700 1 -- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 msgr2=0x7f747c1340b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:35.886 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.759+0000 7f7482d53700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c1340b0 secure :-1 s=READY pgs=121 cs=0 l=1 rx=0x7f74740032f0 tx=0x7f7474007e00).stop 2020-05-18T21:40:35.886 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.759+0000 7f7482d53700 1 -- 172.21.15.196:0/869456078 shutdown_connections 2020-05-18T21:40:35.887 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.759+0000 7f7482d53700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747c13e1a0 0x7f747c1340b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:35.887 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.759+0000 7f7482d53700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f745c04f9f0 0x7f745c051eb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:35.887 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.759+0000 7f7482d53700 1 --2- 172.21.15.196:0/869456078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f747c05b210 0x7f747c0aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:35.888 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.759+0000 7f7482d53700 1 -- 172.21.15.196:0/869456078 >> 172.21.15.196:0/869456078 conn(0x7f747c060640 msgr2=0x7f747c0a5160 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:35.888 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.759+0000 7f7482d53700 1 -- 172.21.15.196:0/869456078 shutdown_connections 2020-05-18T21:40:35.888 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.759+0000 7f7482d53700 1 -- 172.21.15.196:0/869456078 wait complete. 2020-05-18T21:40:35.889 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-05-18T21:40:35.889 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-05-18T21:40:35.890 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mgr dump --format=json 2020-05-18T21:40:35.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe05e36e700 1 Processor -- start 2020-05-18T21:40:35.998 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe05e36e700 1 -- start start 2020-05-18T21:40:36.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe05e36e700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0580a5fd0 0x7fe058134cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:36.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe05e36e700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe058137bb0 0x7fe05812e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:36.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe05e36e700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe058138250 con 0x7fe0580ab910 2020-05-18T21:40:36.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe05e36e700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe058138060 con 0x7fe058137bb0 2020-05-18T21:40:36.002 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe05e36e700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0580a66d0 con 0x7fe0580a5fd0 2020-05-18T21:40:36.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe0580ab910 0x7fe0581345e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43202/0 (socket says 172.21.15.196:43202) 2020-05-18T21:40:36.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 -- 172.21.15.196:0/3443730681 learned_addr learned my addr 172.21.15.196:0/3443730681 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:36.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe0577fe700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0580a5fd0 0x7fe058134cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe05c90b700 1 --2- 172.21.15.196:0/3443730681 >> v2:172.21.15.80:3300/0 conn(0x7fe058137bb0 0x7fe05812e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.003 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649110892 0 0) 0x7fe058138250 con 0x7fe0580ab910 2020-05-18T21:40:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe038003270 con 0x7fe0580ab910 2020-05-18T21:40:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1365669896 0 0) 0x7fe038003270 con 0x7fe0580ab910 2020-05-18T21:40:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0580a5fd0 msgr2=0x7fe058134cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0580a5fd0 0x7fe058134cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 >> v2:172.21.15.80:3300/0 conn(0x7fe058137bb0 msgr2=0x7fe05812e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:36.005 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 --2- 172.21.15.196:0/3443730681 >> v2:172.21.15.80:3300/0 conn(0x7fe058137bb0 0x7fe05812e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:36.005 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0581384f0 con 0x7fe0580ab910 2020-05-18T21:40:36.005 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7fe040001710 con 0x7fe0580ab910 2020-05-18T21:40:36.005 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 >> v1:172.21.15.196:6789/0 conn(0x7fe0580ab910 legacy=0x7fe0581345e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.005 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe038003760 0x7fe038003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:36.006 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe038003fe0 0x7fe038004510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:36.006 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0580a5fd0 0x7fe038004970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:36.006 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0580a66d0 con 0x7fe038003fe0 2020-05-18T21:40:36.006 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe038004dd0 con 0x7fe038003760 2020-05-18T21:40:36.006 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe038004f90 con 0x7fe0580a5fd0 2020-05-18T21:40:36.006 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe038003760 0x7fe038003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe0577fe700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe038003fe0 0x7fe038004510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe05c90b700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0580a5fd0 0x7fe038004970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 -- 172.21.15.196:0/3443730681 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0580a5fd0 msgr2=0x7fe038004970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0580a5fd0 0x7fe038004970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 -- 172.21.15.196:0/3443730681 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe038003fe0 msgr2=0x7fe038004510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:36.008 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe038003fe0 0x7fe038004510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:36.008 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 -- 172.21.15.196:0/3443730681 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe038003270 con 0x7fe038003760 2020-05-18T21:40:36.008 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe0577fe700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe038003fe0 0x7fe038004510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_reply_more state changed! 2020-05-18T21:40:36.008 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe038003760 0x7fe038003b80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fe038004480 tx=0x7fe058135150).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.008 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe038003760 0x7fe038003b80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fe038004480 tx=0x7fe058135150).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.009 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe038003760 0x7fe038003b80 secure :-1 s=READY pgs=123 cs=0 l=1 rx=0x7fe038004480 tx=0x7fe058135150).ready entity=mon.1 client_cookie=1edaff2b7be16660 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.009 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe038003760 0x7fe038003b80 secure :-1 s=THROTTLE_DONE pgs=123 cs=0 l=1 rx=0x7fe038004480 tx=0x7fe058135150).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.009 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe04000ce20 con 0x7fe038003760 2020-05-18T21:40:36.009 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe038003760 0x7fe038003b80 secure :-1 s=THROTTLE_DONE pgs=123 cs=0 l=1 rx=0x7fe038004480 tx=0x7fe058135150).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.009 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04000e370 con 0x7fe038003760 2020-05-18T21:40:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe057fff700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe038003760 0x7fe038003b80 secure :-1 s=THROTTLE_DONE pgs=123 cs=0 l=1 rx=0x7fe038004480 tx=0x7fe058135150).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe056ffd700 1 -- 172.21.15.196:0/3443730681 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe04000e710 con 0x7fe038003760 2020-05-18T21:40:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:35.999+0000 7fe05e36e700 1 -- 172.21.15.196:0/3443730681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe038003760 msgr2=0x7fe038003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.003+0000 7fe05e36e700 1 --2- 172.21.15.196:0/3443730681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe038003760 0x7fe038003b80 secure :-1 s=READY pgs=123 cs=0 l=1 rx=0x7fe038004480 tx=0x7fe058135150).stop 2020-05-18T21:40:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.003+0000 7fe05c90b700 1 -- 172.21.15.196:0/3443730681 reap_dead start 2020-05-18T21:40:36.014 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.003+0000 7fe05e36e700 1 -- 172.21.15.196:0/3443730681 shutdown_connections 2020-05-18T21:40:36.015 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.003+0000 7fe05e36e700 1 -- 172.21.15.196:0/3443730681 >> 172.21.15.196:0/3443730681 conn(0x7fe058060fe0 msgr2=0x7fe0580aaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:36.015 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.003+0000 7fe05e36e700 1 -- 172.21.15.196:0/3443730681 shutdown_connections 2020-05-18T21:40:36.015 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.003+0000 7fe05e36e700 1 -- 172.21.15.196:0/3443730681 wait complete. 2020-05-18T21:40:36.015 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.003+0000 7fe05e36e700 1 Processor -- start 2020-05-18T21:40:36.015 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.003+0000 7fe05e36e700 1 -- start start 2020-05-18T21:40:36.016 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05e36e700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe05812f6d0 0x7fe058141730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.016 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05e36e700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe058141b90 0x7fe05814b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.016 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05e36e700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05809ea70 con 0x7fe0580a2110 2020-05-18T21:40:36.016 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05e36e700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0580a52f0 con 0x7fe058141b90 2020-05-18T21:40:36.016 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05e36e700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe058131430 con 0x7fe05812f6d0 2020-05-18T21:40:36.017 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe057fff700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe0580a2110 0x7fe058141020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43214/0 (socket says 172.21.15.196:43214) 2020-05-18T21:40:36.017 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe057fff700 1 -- 172.21.15.196:0/1538038901 learned_addr learned my addr 172.21.15.196:0/1538038901 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:36.017 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> v2:172.21.15.80:3300/0 conn(0x7fe058141b90 0x7fe05814b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.017 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0577fe700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe05812f6d0 0x7fe058141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.017 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79935617 0 0) 0x7fe05809ea70 con 0x7fe0580a2110 2020-05-18T21:40:36.018 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe03c002fe0 con 0x7fe0580a2110 2020-05-18T21:40:36.018 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 3813002440 0 0) 0x7fe03c002fe0 con 0x7fe0580a2110 2020-05-18T21:40:36.018 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 --> v1:172.21.15.196:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe05809ea70 con 0x7fe0580a2110 2020-05-18T21:40:36.018 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7fe04000ce20 con 0x7fe0580a2110 2020-05-18T21:40:36.018 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe05812f6d0 msgr2=0x7fe058141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.019 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe05812f6d0 0x7fe058141730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.019 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 >> v2:172.21.15.80:3300/0 conn(0x7fe058141b90 msgr2=0x7fe05814b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.019 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 --2- 172.21.15.196:0/1538038901 >> v2:172.21.15.80:3300/0 conn(0x7fe058141b90 0x7fe05814b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.019 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 >> v1:172.21.15.196:6789/0 conn(0x7fe0580a2110 legacy=0x7fe058141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.019 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe05812f6d0 0x7fe03c004a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.019 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe03c004ee0 0x7fe03c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.020 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.020 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0580a52f0 con 0x7fe03c004ee0 2020-05-18T21:40:36.020 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe058131430 con 0x7fe03c003e00 2020-05-18T21:40:36.020 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0053c0 con 0x7fe05812f6d0 2020-05-18T21:40:36.020 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe057fff700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe05812f6d0 0x7fe03c004a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.021 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0577fe700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe03c004ee0 0x7fe03c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.021 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.021 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05c90b700 1 -- 172.21.15.196:0/1538038901 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe05812f6d0 msgr2=0x7fe03c004a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.021 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe05812f6d0 0x7fe03c004a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.021 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05c90b700 1 -- 172.21.15.196:0/1538038901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe03c004ee0 msgr2=0x7fe03c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.022 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe03c004ee0 0x7fe03c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.022 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05c90b700 1 -- 172.21.15.196:0/1538038901 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05812fb30 con 0x7fe03c003e00 2020-05-18T21:40:36.022 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05e36e700 1 -- 172.21.15.196:0/1538038901 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0581309f0 con 0x7fe03c003e00 2020-05-18T21:40:36.022 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe040001970 tx=0x7fe050010bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.022 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05e36e700 1 -- 172.21.15.196:0/1538038901 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05812f230 con 0x7fe03c003e00 2020-05-18T21:40:36.023 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe040001970 tx=0x7fe050010bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.023 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 secure :-1 s=READY pgs=125 cs=0 l=1 rx=0x7fe040001970 tx=0x7fe050010bf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.023 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fe040001970 tx=0x7fe050010bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.023 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe050007370 con 0x7fe03c003e00 2020-05-18T21:40:36.023 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fe040001970 tx=0x7fe050010bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.023 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fe040001970 tx=0x7fe050010bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.024 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0500074d0 con 0x7fe03c003e00 2020-05-18T21:40:36.024 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.007+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe050005bf0 con 0x7fe03c003e00 2020-05-18T21:40:36.024 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.011+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fe040001970 tx=0x7fe050010bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.024 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.011+0000 7fe05e36e700 1 -- 172.21.15.196:0/1538038901 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe030003380 con 0x7fe03c003e00 2020-05-18T21:40:36.024 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.011+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fe050005d50 con 0x7fe03c003e00 2020-05-18T21:40:36.025 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.011+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fe040001970 tx=0x7fe050010bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.025 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.011+0000 7fe0557fa700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe03c057ae0 0x7fe03c059fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.025 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.011+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe0500637d0 con 0x7fe03c003e00 2020-05-18T21:40:36.026 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.011+0000 7fe057fff700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe03c057ae0 0x7fe03c059fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.026 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.011+0000 7fe057fff700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe03c057ae0 0x7fe03c059fa0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe040001110 tx=0x7fe058135150).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.026 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.011+0000 7fe057fff700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe03c057ae0 0x7fe03c059fa0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe040001110 tx=0x7fe058135150).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.026 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.011+0000 7fe057fff700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe03c057ae0 0x7fe03c059fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rx=0x7fe040001110 tx=0x7fe058135150).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.027 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.015+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fe040001970 tx=0x7fe050010bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.027 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.015+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7fe05003db60 con 0x7fe03c003e00 2020-05-18T21:40:36.160 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.159+0000 7fe05e36e700 1 -- 172.21.15.196:0/1538038901 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe030002210 con 0x7fe03c003e00 2020-05-18T21:40:36.166 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.167+0000 7fe05c90b700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fe040001970 tx=0x7fe050010bf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.166 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.167+0000 7fe0557fa700 1 -- 172.21.15.196:0/1538038901 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+123723 (secure 0 0 0) 0x7fe050041180 con 0x7fe03c003e00 2020-05-18T21:40:36.168 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:36.170 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.171+0000 7fe05e36e700 1 -- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe03c057ae0 msgr2=0x7fe03c059fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.170 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.171+0000 7fe05e36e700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe03c057ae0 0x7fe03c059fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rx=0x7fe040001110 tx=0x7fe058135150).stop 2020-05-18T21:40:36.170 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.171+0000 7fe05e36e700 1 -- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 msgr2=0x7fe03c004280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.171 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.171+0000 7fe05c90b700 1 -- 172.21.15.196:0/1538038901 reap_dead start 2020-05-18T21:40:36.171 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.171+0000 7fe05e36e700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 secure :-1 s=READY pgs=125 cs=0 l=1 rx=0x7fe040001970 tx=0x7fe050010bf0).stop 2020-05-18T21:40:36.171 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.171+0000 7fe05e36e700 1 -- 172.21.15.196:0/1538038901 shutdown_connections 2020-05-18T21:40:36.171 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.171+0000 7fe05e36e700 1 --2- 172.21.15.196:0/1538038901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe03c003e00 0x7fe03c004280 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.171 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.171+0000 7fe05e36e700 1 -- 172.21.15.196:0/1538038901 >> 172.21.15.196:0/1538038901 conn(0x7fe058060fe0 msgr2=0x7fe0580a5770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:36.172 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.171+0000 7fe05e36e700 1 -- 172.21.15.196:0/1538038901 shutdown_connections 2020-05-18T21:40:36.172 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.171+0000 7fe05e36e700 1 -- 172.21.15.196:0/1538038901 wait complete. 2020-05-18T21:40:36.185 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":4,"active_gid":4105,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":12657},{"type":"v1","addr":"172.21.15.196:6833","nonce":12657}]},"active_addr":"172.21.15.196:6833/12657","active_change":"2020-05-18T21:40:19.737167+0000","active_mgr_features":4540138292837744639,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540138292837744639,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","osd_support","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","osd_support","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.196:0","nonce":201964452}]},{"addrvec":[{"type":"v2","addr":"172.21.15.196:0","nonce":22629481}]},{"addrvec":[{"type":"v2","addr":"172.21.15.196:0","nonce":929928857}]}]}} 2020-05-18T21:40:36.189 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-05-18T21:40:36.190 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-05-18T21:40:36.190 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-05-18T21:40:36.291 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.291+0000 7fc54a17b700 1 Processor -- start 2020-05-18T21:40:36.291 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.291+0000 7fc54a17b700 1 -- start start 2020-05-18T21:40:36.294 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.291+0000 7fc54a17b700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc5440a7220 0x7fc5440a3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:36.294 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.291+0000 7fc54a17b700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54409e0b0 con 0x7fc5440a3dd0 2020-05-18T21:40:36.294 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.291+0000 7fc54a17b700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5440a20f0 con 0x7fc544140840 2020-05-18T21:40:36.294 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.291+0000 7fc54a17b700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5440a3580 con 0x7fc5440a7220 2020-05-18T21:40:36.294 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.291+0000 7fc549179700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc5440a7220 0x7fc5440a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.295 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.291+0000 7fc54997a700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc544140840 0x7fc544146fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43230/0 (socket says 172.21.15.196:43230) 2020-05-18T21:40:36.295 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.291+0000 7fc54997a700 1 -- 172.21.15.196:0/1111535733 learned_addr learned my addr 172.21.15.196:0/1111535733 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:36.295 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.291+0000 7fc549179700 1 -- 172.21.15.196:0/1111535733 >> v1:172.21.15.196:6789/0 conn(0x7fc544140840 legacy=0x7fc544146fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.295 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.291+0000 7fc543fff700 1 -- 172.21.15.196:0/1111535733 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666189991 0 0) 0x7fc54409e0b0 con 0x7fc5440a3dd0 2020-05-18T21:40:36.295 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.291+0000 7fc549179700 1 -- 172.21.15.196:0/1111535733 >> v1:172.21.15.80:6789/0 conn(0x7fc5440a3dd0 legacy=0x7fc544140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.296 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.291+0000 7fc549179700 1 -- 172.21.15.196:0/1111535733 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5440a6a60 con 0x7fc5440a7220 2020-05-18T21:40:36.296 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc549179700 1 --2- 172.21.15.196:0/1111535733 >> v2:172.21.15.196:3300/0 conn(0x7fc5440a7220 0x7fc5440a3970 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.296 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc549179700 1 --2- 172.21.15.196:0/1111535733 >> v2:172.21.15.196:3300/0 conn(0x7fc5440a7220 0x7fc5440a3970 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.296 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc549179700 1 --2- 172.21.15.196:0/1111535733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a7220 0x7fc5440a3970 secure :-1 s=READY pgs=127 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).ready entity=mon.1 client_cookie=f59dc3bb3ac754c7 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.296 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc549179700 1 --2- 172.21.15.196:0/1111535733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a7220 0x7fc5440a3970 secure :-1 s=THROTTLE_DONE pgs=127 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.297 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc543fff700 1 -- 172.21.15.196:0/1111535733 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc538008d30 con 0x7fc5440a7220 2020-05-18T21:40:36.297 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc549179700 1 --2- 172.21.15.196:0/1111535733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a7220 0x7fc5440a3970 secure :-1 s=THROTTLE_DONE pgs=127 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.297 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc549179700 1 --2- 172.21.15.196:0/1111535733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a7220 0x7fc5440a3970 secure :-1 s=THROTTLE_DONE pgs=127 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.297 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc543fff700 1 -- 172.21.15.196:0/1111535733 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc538008e90 con 0x7fc5440a7220 2020-05-18T21:40:36.297 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc543fff700 1 -- 172.21.15.196:0/1111535733 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc538011a20 con 0x7fc5440a7220 2020-05-18T21:40:36.298 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 -- 172.21.15.196:0/1111535733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a7220 msgr2=0x7fc5440a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.298 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 --2- 172.21.15.196:0/1111535733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a7220 0x7fc5440a3970 secure :-1 s=READY pgs=127 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).stop 2020-05-18T21:40:36.301 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 -- 172.21.15.196:0/1111535733 shutdown_connections 2020-05-18T21:40:36.301 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 --2- 172.21.15.196:0/1111535733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a7220 0x7fc5440a3970 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.301 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 -- 172.21.15.196:0/1111535733 >> 172.21.15.196:0/1111535733 conn(0x7fc54405a7c0 msgr2=0x7fc54405abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:36.301 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 -- 172.21.15.196:0/1111535733 shutdown_connections 2020-05-18T21:40:36.302 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 -- 172.21.15.196:0/1111535733 wait complete. 2020-05-18T21:40:36.302 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 Processor -- start 2020-05-18T21:40:36.302 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 -- start start 2020-05-18T21:40:36.302 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc5440a3dd0 0x7fc54409d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.302 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc544140840 0x7fc54409cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.302 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54409ab60 con 0x7fc5440a7220 2020-05-18T21:40:36.303 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5440a0ef0 con 0x7fc544140840 2020-05-18T21:40:36.303 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54a17b700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54409adc0 con 0x7fc5440a3dd0 2020-05-18T21:40:36.303 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc549179700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc5440a3dd0 0x7fc54409d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.303 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc549179700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc5440a3dd0 0x7fc54409d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34474/0 (socket says 172.21.15.196:34474) 2020-05-18T21:40:36.303 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc549179700 1 -- 172.21.15.196:0/3002056429 learned_addr learned my addr 172.21.15.196:0/3002056429 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:36.304 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.295+0000 7fc54997a700 1 --2- 172.21.15.196:0/3002056429 >> v2:172.21.15.80:3300/0 conn(0x7fc544140840 0x7fc54409cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.304 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc549179700 1 -- 172.21.15.196:0/3002056429 >> v2:172.21.15.80:3300/0 conn(0x7fc544140840 msgr2=0x7fc54409cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.304 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc549179700 1 --2- 172.21.15.196:0/3002056429 >> v2:172.21.15.80:3300/0 conn(0x7fc544140840 0x7fc54409cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.304 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc549179700 1 -- 172.21.15.196:0/3002056429 >> v1:172.21.15.80:6789/0 conn(0x7fc5440a7220 legacy=0x7fc544099790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.304 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc549179700 1 -- 172.21.15.196:0/3002056429 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54409b260 con 0x7fc5440a3dd0 2020-05-18T21:40:36.305 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc54997a700 1 --2- 172.21.15.196:0/3002056429 >> v2:172.21.15.80:3300/0 conn(0x7fc544140840 0x7fc54409cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-05-18T21:40:36.305 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc549179700 1 --2- 172.21.15.196:0/3002056429 >> v2:172.21.15.196:3300/0 conn(0x7fc5440a3dd0 0x7fc54409d5a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.305 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc549179700 1 --2- 172.21.15.196:0/3002056429 >> v2:172.21.15.196:3300/0 conn(0x7fc5440a3dd0 0x7fc54409d5a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.305 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc549179700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a3dd0 0x7fc54409d5a0 secure :-1 s=READY pgs=128 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.305 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc549179700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a3dd0 0x7fc54409d5a0 secure :-1 s=THROTTLE_DONE pgs=128 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.306 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc549179700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a3dd0 0x7fc54409d5a0 secure :-1 s=THROTTLE_DONE pgs=128 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.306 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc5427fc700 1 -- 172.21.15.196:0/3002056429 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc53800f850 con 0x7fc5440a3dd0 2020-05-18T21:40:36.306 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc549179700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a3dd0 0x7fc54409d5a0 secure :-1 s=THROTTLE_DONE pgs=128 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.306 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc5427fc700 1 -- 172.21.15.196:0/3002056429 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc538007360 con 0x7fc5440a3dd0 2020-05-18T21:40:36.306 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc54a17b700 1 -- 172.21.15.196:0/3002056429 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54409b4b0 con 0x7fc5440a3dd0 2020-05-18T21:40:36.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc5427fc700 1 -- 172.21.15.196:0/3002056429 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc538005700 con 0x7fc5440a3dd0 2020-05-18T21:40:36.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc54a17b700 1 -- 172.21.15.196:0/3002056429 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5440a0a00 con 0x7fc5440a3dd0 2020-05-18T21:40:36.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc549179700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a3dd0 0x7fc54409d5a0 secure :-1 s=THROTTLE_DONE pgs=128 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc54a17b700 1 -- 172.21.15.196:0/3002056429 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5440a0d10 con 0x7fc5440a3dd0 2020-05-18T21:40:36.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc5427fc700 1 -- 172.21.15.196:0/3002056429 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fc538005930 con 0x7fc5440a3dd0 2020-05-18T21:40:36.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc549179700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a3dd0 0x7fc54409d5a0 secure :-1 s=THROTTLE_DONE pgs=128 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.308 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc5427fc700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc52404f9c0 0x7fc524051e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.308 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc5427fc700 1 -- 172.21.15.196:0/3002056429 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc5380643a0 con 0x7fc5440a3dd0 2020-05-18T21:40:36.308 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.299+0000 7fc548978700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc52404f9c0 0x7fc524051e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.303+0000 7fc548978700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc52404f9c0 0x7fc524051e80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc5440a8e60 tx=0x7fc53c002a10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.303+0000 7fc548978700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc52404f9c0 0x7fc524051e80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc5440a8e60 tx=0x7fc53c002a10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.303+0000 7fc548978700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc52404f9c0 0x7fc524051e80 secure :-1 s=READY pgs=47 cs=0 l=1 rx=0x7fc5440a8e60 tx=0x7fc53c002a10).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.307+0000 7fc549179700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a3dd0 0x7fc54409d5a0 secure :-1 s=THROTTLE_DONE pgs=128 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.310 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.307+0000 7fc5427fc700 1 -- 172.21.15.196:0/3002056429 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7fc53803e850 con 0x7fc5440a3dd0 2020-05-18T21:40:36.421 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.419+0000 7fc54a17b700 1 -- 172.21.15.196:0/3002056429 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5440569e0 con 0x7fc5440a3dd0 2020-05-18T21:40:36.422 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.423+0000 7fc549179700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a3dd0 0x7fc54409d5a0 secure :-1 s=THROTTLE_DONE pgs=128 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.423 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.423+0000 7fc5427fc700 1 -- 172.21.15.196:0/3002056429 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16089 (secure 0 0 0) 0x7fc538041e70 con 0x7fc5440a3dd0 2020-05-18T21:40:36.423 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:36.431 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"fsid":"d048673a-3d91-4cc6-882d-e9cc70e7cd6c","created":"2020-05-18T21:40:15.911738+0000","modified":"2020-05-18T21:40:33.211283+0000","last_up_change":"2020-05-18T21:40:22.762927+0000","last_in_change":"2020-05-18T21:40:22.762927+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-05-18T21:40:19.808941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-05-18T21:40:26.944311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-05-18T21:40:30.135661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-05-18T21:40:31.183488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8db63a0a-61dc-42b5-9559-fddcf436cab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":12771},{"type":"v1","addr":"172.21.15.196:6801","nonce":12771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":12771},{"type":"v1","addr":"172.21.15.196:6803","nonce":12771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":12771},{"type":"v1","addr":"172.21.15.196:6807","nonce":12771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":12771},{"type":"v1","addr":"172.21.15.196:6805","nonce":12771}]},"public_addr":"172.21.15.196:6801/12771","cluster_addr":"172.21.15.196:6803/12771","heartbeat_back_addr":"172.21.15.196:6807/12771","heartbeat_front_addr":"172.21.15.196:6805/12771","state":["exists","up"]},{"osd":1,"uuid":"6773b0c3-9f49-42a2-b75b-c10c734ff086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":12772},{"type":"v1","addr":"172.21.15.196:6817","nonce":12772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":12772},{"type":"v1","addr":"172.21.15.196:6819","nonce":12772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":12772},{"type":"v1","addr":"172.21.15.196:6823","nonce":12772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":12772},{"type":"v1","addr":"172.21.15.196:6821","nonce":12772}]},"public_addr":"172.21.15.196:6817/12772","cluster_addr":"172.21.15.196:6819/12772","heartbeat_back_addr":"172.21.15.196:6823/12772","heartbeat_front_addr":"172.21.15.196:6821/12772","state":["exists","up"]},{"osd":2,"uuid":"ac1f2e05-9bb7-4768-b712-10e4d2ea4154","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":12773},{"type":"v1","addr":"172.21.15.196:6809","nonce":12773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":12773},{"type":"v1","addr":"172.21.15.196:6811","nonce":12773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":12773},{"type":"v1","addr":"172.21.15.196:6815","nonce":12773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":12773},{"type":"v1","addr":"172.21.15.196:6813","nonce":12773}]},"public_addr":"172.21.15.196:6809/12773","cluster_addr":"172.21.15.196:6811/12773","heartbeat_back_addr":"172.21.15.196:6815/12773","heartbeat_front_addr":"172.21.15.196:6813/12773","state":["exists","up"]},{"osd":3,"uuid":"bae5ab40-235b-4e0f-8d0d-524afd099b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":12775},{"type":"v1","addr":"172.21.15.196:6825","nonce":12775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":12775},{"type":"v1","addr":"172.21.15.196:6827","nonce":12775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":12775},{"type":"v1","addr":"172.21.15.196:6831","nonce":12775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":12775},{"type":"v1","addr":"172.21.15.196:6829","nonce":12775}]},"public_addr":"172.21.15.196:6825/12775","cluster_addr":"172.21.15.196:6827/12775","heartbeat_back_addr":"172.21.15.196:6831/12775","heartbeat_front_addr":"172.21.15.196:6829/12775","state":["exists","up"]},{"osd":4,"uuid":"275fc52c-58e4-4f35-a0cb-e117fe36bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":13040},{"type":"v1","addr":"172.21.15.80:6819","nonce":13040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6821","nonce":13040},{"type":"v1","addr":"172.21.15.80:6823","nonce":13040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6829","nonce":13040},{"type":"v1","addr":"172.21.15.80:6831","nonce":13040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":13040},{"type":"v1","addr":"172.21.15.80:6827","nonce":13040}]},"public_addr":"172.21.15.80:6819/13040","cluster_addr":"172.21.15.80:6823/13040","heartbeat_back_addr":"172.21.15.80:6831/13040","heartbeat_front_addr":"172.21.15.80:6827/13040","state":["exists","up"]},{"osd":5,"uuid":"9795b89c-e2f8-4abf-b9c7-796727a57002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":13037},{"type":"v1","addr":"172.21.15.80:6818","nonce":13037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":13037},{"type":"v1","addr":"172.21.15.80:6822","nonce":13037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":13037},{"type":"v1","addr":"172.21.15.80:6830","nonce":13037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":13037},{"type":"v1","addr":"172.21.15.80:6826","nonce":13037}]},"public_addr":"172.21.15.80:6818/13037","cluster_addr":"172.21.15.80:6822/13037","heartbeat_back_addr":"172.21.15.80:6830/13037","heartbeat_front_addr":"172.21.15.80:6826/13037","state":["exists","up"]},{"osd":6,"uuid":"e42df2e8-9cc5-4c7e-a243-bd18ccfbd002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":13039},{"type":"v1","addr":"172.21.15.80:6804","nonce":13039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":13039},{"type":"v1","addr":"172.21.15.80:6808","nonce":13039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":13039},{"type":"v1","addr":"172.21.15.80:6813","nonce":13039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":13039},{"type":"v1","addr":"172.21.15.80:6810","nonce":13039}]},"public_addr":"172.21.15.80:6804/13039","cluster_addr":"172.21.15.80:6808/13039","heartbeat_back_addr":"172.21.15.80:6813/13039","heartbeat_front_addr":"172.21.15.80:6810/13039","state":["exists","up"]},{"osd":7,"uuid":"ebd6c938-beff-4668-b70e-076ed22bb8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":13038},{"type":"v1","addr":"172.21.15.80:6801","nonce":13038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":13038},{"type":"v1","addr":"172.21.15.80:6805","nonce":13038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":13038},{"type":"v1","addr":"172.21.15.80:6817","nonce":13038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":13038},{"type":"v1","addr":"172.21.15.80:6811","nonce":13038}]},"public_addr":"172.21.15.80:6801/13038","cluster_addr":"172.21.15.80:6805/13038","heartbeat_back_addr":"172.21.15.80:6817/13038","heartbeat_front_addr":"172.21.15.80:6811/13038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.134515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.439836+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.476825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.176642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.452301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.427183+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.230250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.210168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-18T21:40:36.432 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.423+0000 7fc54a17b700 1 -- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc52404f9c0 msgr2=0x7fc524051e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.432 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.423+0000 7fc54a17b700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc52404f9c0 0x7fc524051e80 secure :-1 s=READY pgs=47 cs=0 l=1 rx=0x7fc5440a8e60 tx=0x7fc53c002a10).stop 2020-05-18T21:40:36.433 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.423+0000 7fc54a17b700 1 -- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a3dd0 msgr2=0x7fc54409d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.433 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.423+0000 7fc54a17b700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a3dd0 0x7fc54409d5a0 secure :-1 s=READY pgs=128 cs=0 l=1 rx=0x7fc534001110 tx=0x7fc538007a80).stop 2020-05-18T21:40:36.433 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.423+0000 7fc54a17b700 1 -- 172.21.15.196:0/3002056429 shutdown_connections 2020-05-18T21:40:36.433 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.423+0000 7fc54a17b700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc52404f9c0 0x7fc524051e80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.433 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.423+0000 7fc54a17b700 1 --2- 172.21.15.196:0/3002056429 >> v2:172.21.15.80:3300/0 conn(0x7fc544140840 0x7fc54409cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.434 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.423+0000 7fc54a17b700 1 --2- 172.21.15.196:0/3002056429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5440a3dd0 0x7fc54409d5a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.434 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.423+0000 7fc54a17b700 1 -- 172.21.15.196:0/3002056429 >> 172.21.15.196:0/3002056429 conn(0x7fc54405a7c0 msgr2=0x7fc54405aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:36.434 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.423+0000 7fc54a17b700 1 -- 172.21.15.196:0/3002056429 shutdown_connections 2020-05-18T21:40:36.434 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.423+0000 7fc54a17b700 1 -- 172.21.15.196:0/3002056429 wait complete. 2020-05-18T21:40:36.435 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-05-18T21:40:36.436 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-05-18T21:40:36.538 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a225cd700 1 Processor -- start 2020-05-18T21:40:36.538 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a225cd700 1 -- start start 2020-05-18T21:40:36.540 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a225cd700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c065f70 0x7f5a1c0aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:36.540 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a225cd700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5a1c066910 0x7f5a1c05a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:36.541 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a225cd700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c05b2e0 con 0x7f5a1c05ae30 2020-05-18T21:40:36.541 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a225cd700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a1c0a9d60 con 0x7f5a1c066910 2020-05-18T21:40:36.541 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a225cd700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0a9360 con 0x7f5a1c065f70 2020-05-18T21:40:36.541 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a20dca700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5a1c066910 0x7f5a1c05a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.541 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a215cb700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c065f70 0x7f5a1c0aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.542 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a20dca700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5a1c066910 0x7f5a1c05a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.196:38708/0 (socket says 172.21.15.196:38708) 2020-05-18T21:40:36.542 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a20dca700 1 -- 172.21.15.196:0/3177921613 learned_addr learned my addr 172.21.15.196:0/3177921613 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:36.542 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a215cb700 1 --2- 172.21.15.196:0/3177921613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c065f70 0x7f5a1c0aafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.196:38314/0 (socket says 172.21.15.196:38314) 2020-05-18T21:40:36.542 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a215cb700 1 -- 172.21.15.196:0/3177921613 >> v2:172.21.15.80:3300/0 conn(0x7f5a1c066910 msgr2=0x7f5a1c05a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:36.542 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a215cb700 1 --2- 172.21.15.196:0/3177921613 >> v2:172.21.15.80:3300/0 conn(0x7f5a1c066910 0x7f5a1c05a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:36.543 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a215cb700 1 -- 172.21.15.196:0/3177921613 >> v1:172.21.15.80:6789/0 conn(0x7f5a1c05ae30 legacy=0x7f5a1c13b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.543 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a215cb700 1 -- 172.21.15.196:0/3177921613 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c0aab80 con 0x7f5a1c065f70 2020-05-18T21:40:36.543 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a20dca700 1 --2- 172.21.15.196:0/3177921613 >> v2:172.21.15.80:3300/0 conn(0x7f5a1c066910 0x7f5a1c05a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:36.543 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a215cb700 1 --2- 172.21.15.196:0/3177921613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c065f70 0x7f5a1c0aafb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f5a10002f70 tx=0x7f5a10007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.543 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a215cb700 1 --2- 172.21.15.196:0/3177921613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c065f70 0x7f5a1c0aafb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f5a10002f70 tx=0x7f5a10007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.544 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a215cb700 1 --2- 172.21.15.196:0/3177921613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c065f70 0x7f5a1c0aafb0 secure :-1 s=READY pgs=22 cs=0 l=1 rx=0x7f5a10002f70 tx=0x7f5a10007a80).ready entity=mon.2 client_cookie=37206ead8803827f server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.544 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a215cb700 1 --2- 172.21.15.196:0/3177921613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c065f70 0x7f5a1c0aafb0 secure :-1 s=THROTTLE_DONE pgs=22 cs=0 l=1 rx=0x7f5a10002f70 tx=0x7f5a10007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.544 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a215cb700 1 --2- 172.21.15.196:0/3177921613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c065f70 0x7f5a1c0aafb0 secure :-1 s=THROTTLE_DONE pgs=22 cs=0 l=1 rx=0x7f5a10002f70 tx=0x7f5a10007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.544 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a0ffff700 1 -- 172.21.15.196:0/3177921613 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a10008e80 con 0x7f5a1c065f70 2020-05-18T21:40:36.544 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a215cb700 1 --2- 172.21.15.196:0/3177921613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c065f70 0x7f5a1c0aafb0 secure :-1 s=THROTTLE_DONE pgs=22 cs=0 l=1 rx=0x7f5a10002f70 tx=0x7f5a10007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.544 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a0ffff700 1 -- 172.21.15.196:0/3177921613 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a100115f0 con 0x7f5a1c065f70 2020-05-18T21:40:36.545 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a225cd700 1 -- 172.21.15.196:0/3177921613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c065f70 msgr2=0x7f5a1c0aafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.545 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a225cd700 1 --2- 172.21.15.196:0/3177921613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c065f70 0x7f5a1c0aafb0 secure :-1 s=READY pgs=22 cs=0 l=1 rx=0x7f5a10002f70 tx=0x7f5a10007a80).stop 2020-05-18T21:40:36.545 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.539+0000 7f5a0ffff700 1 -- 172.21.15.196:0/3177921613 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a10011a10 con 0x7f5a1c065f70 2020-05-18T21:40:36.548 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a225cd700 1 -- 172.21.15.196:0/3177921613 shutdown_connections 2020-05-18T21:40:36.548 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a225cd700 1 --2- 172.21.15.196:0/3177921613 >> v2:172.21.15.80:3300/0 conn(0x7f5a1c066910 0x7f5a1c05a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:36.549 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a225cd700 1 --2- 172.21.15.196:0/3177921613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c065f70 0x7f5a1c0aafb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.549 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a225cd700 1 -- 172.21.15.196:0/3177921613 >> 172.21.15.196:0/3177921613 conn(0x7f5a1c0ae900 msgr2=0x7f5a1c0aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:36.549 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a225cd700 1 -- 172.21.15.196:0/3177921613 shutdown_connections 2020-05-18T21:40:36.549 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a225cd700 1 -- 172.21.15.196:0/3177921613 wait complete. 2020-05-18T21:40:36.549 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a225cd700 1 Processor -- start 2020-05-18T21:40:36.549 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a225cd700 1 -- start start 2020-05-18T21:40:36.550 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a225cd700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.550 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a225cd700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c0a43e0 con 0x7f5a1c065f70 2020-05-18T21:40:36.550 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a225cd700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c0a4680 con 0x7f5a1c05ae30 2020-05-18T21:40:36.550 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a225cd700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0a2570 con 0x7f5a1c066910 2020-05-18T21:40:36.550 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a215cb700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f5a1c05ae30 0x7f5a1c09f150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43246/0 (socket says 172.21.15.196:43246) 2020-05-18T21:40:36.551 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a215cb700 1 -- 172.21.15.196:0/25111458 learned_addr learned my addr 172.21.15.196:0/25111458 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:36.551 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.543+0000 7f5a21dcc700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.551 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a0effd700 1 -- 172.21.15.196:0/25111458 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445474350 0 0) 0x7f5a1c0a4680 con 0x7f5a1c05ae30 2020-05-18T21:40:36.551 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a0effd700 1 -- 172.21.15.196:0/25111458 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59f8002fe0 con 0x7f5a1c05ae30 2020-05-18T21:40:36.551 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a0effd700 1 -- 172.21.15.196:0/25111458 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797088494 0 0) 0x7f5a1c0a43e0 con 0x7f5a1c065f70 2020-05-18T21:40:36.552 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a0effd700 1 -- 172.21.15.196:0/25111458 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a1c0a4680 con 0x7f5a1c065f70 2020-05-18T21:40:36.552 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a21dcc700 1 -- 172.21.15.196:0/25111458 >> v1:172.21.15.196:6789/0 conn(0x7f5a1c05ae30 legacy=0x7f5a1c09f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.552 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a21dcc700 1 -- 172.21.15.196:0/25111458 >> v1:172.21.15.80:6789/0 conn(0x7f5a1c065f70 legacy=0x7f5a1c0a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.552 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a21dcc700 1 -- 172.21.15.196:0/25111458 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c0a2770 con 0x7f5a1c066910 2020-05-18T21:40:36.552 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a21dcc700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f5a1c060960 tx=0x7f5a18001e70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.553 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a21dcc700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f5a1c060960 tx=0x7f5a18001e70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.553 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a21dcc700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 secure :-1 s=READY pgs=23 cs=0 l=1 rx=0x7f5a1c060960 tx=0x7f5a18001e70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.553 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a21dcc700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7f5a1c060960 tx=0x7f5a18001e70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.553 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a0effd700 1 -- 172.21.15.196:0/25111458 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a1800cd30 con 0x7f5a1c066910 2020-05-18T21:40:36.553 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a225cd700 1 -- 172.21.15.196:0/25111458 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c0a5680 con 0x7f5a1c066910 2020-05-18T21:40:36.553 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a21dcc700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7f5a1c060960 tx=0x7f5a18001e70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.554 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a21dcc700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7f5a1c060960 tx=0x7f5a18001e70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.554 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a0effd700 1 -- 172.21.15.196:0/25111458 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1800ce90 con 0x7f5a1c066910 2020-05-18T21:40:36.554 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a0effd700 1 -- 172.21.15.196:0/25111458 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a18011030 con 0x7f5a1c066910 2020-05-18T21:40:36.554 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a225cd700 1 -- 172.21.15.196:0/25111458 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1c138bf0 con 0x7f5a1c066910 2020-05-18T21:40:36.554 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a21dcc700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7f5a1c060960 tx=0x7f5a18001e70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.555 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a225cd700 1 -- 172.21.15.196:0/25111458 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1c09f950 con 0x7f5a1c066910 2020-05-18T21:40:36.555 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a0effd700 1 -- 172.21.15.196:0/25111458 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f5a18006520 con 0x7f5a1c066910 2020-05-18T21:40:36.555 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a21dcc700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7f5a1c060960 tx=0x7f5a18001e70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.555 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a0effd700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f59f804feb0 0x7f59f8052370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.556 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a0effd700 1 -- 172.21.15.196:0/25111458 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5a18063610 con 0x7f5a1c066910 2020-05-18T21:40:36.556 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.547+0000 7f5a215cb700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f59f804feb0 0x7f59f8052370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.556 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.551+0000 7f5a215cb700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f59f804feb0 0x7f59f8052370 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f5a10008950 tx=0x7f5a10007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.557 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.551+0000 7f5a215cb700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f59f804feb0 0x7f59f8052370 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f5a10008950 tx=0x7f5a10007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.557 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.551+0000 7f5a215cb700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f59f804feb0 0x7f59f8052370 secure :-1 s=READY pgs=48 cs=0 l=1 rx=0x7f5a10008950 tx=0x7f5a10007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.557 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.559+0000 7f5a21dcc700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7f5a1c060960 tx=0x7f5a18001e70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.558 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.559+0000 7f5a0effd700 1 -- 172.21.15.196:0/25111458 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f5a1803d9a0 con 0x7f5a1c066910 2020-05-18T21:40:36.674 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.675+0000 7f5a225cd700 1 -- 172.21.15.196:0/25111458 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a1c0a4810 con 0x7f5a1c066910 2020-05-18T21:40:36.676 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.675+0000 7f5a21dcc700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 secure :-1 s=THROTTLE_DONE pgs=23 cs=0 l=1 rx=0x7f5a1c060960 tx=0x7f5a18001e70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.676 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.675+0000 7f5a0effd700 1 -- 172.21.15.196:0/25111458 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16089 (secure 0 0 0) 0x7f5a18040fc0 con 0x7f5a1c066910 2020-05-18T21:40:36.676 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:36.677 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"fsid":"d048673a-3d91-4cc6-882d-e9cc70e7cd6c","created":"2020-05-18T21:40:15.911738+0000","modified":"2020-05-18T21:40:33.211283+0000","last_up_change":"2020-05-18T21:40:22.762927+0000","last_in_change":"2020-05-18T21:40:22.762927+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-05-18T21:40:19.808941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-05-18T21:40:26.944311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-05-18T21:40:30.135661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-05-18T21:40:31.183488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8db63a0a-61dc-42b5-9559-fddcf436cab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":12771},{"type":"v1","addr":"172.21.15.196:6801","nonce":12771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":12771},{"type":"v1","addr":"172.21.15.196:6803","nonce":12771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":12771},{"type":"v1","addr":"172.21.15.196:6807","nonce":12771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":12771},{"type":"v1","addr":"172.21.15.196:6805","nonce":12771}]},"public_addr":"172.21.15.196:6801/12771","cluster_addr":"172.21.15.196:6803/12771","heartbeat_back_addr":"172.21.15.196:6807/12771","heartbeat_front_addr":"172.21.15.196:6805/12771","state":["exists","up"]},{"osd":1,"uuid":"6773b0c3-9f49-42a2-b75b-c10c734ff086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":12772},{"type":"v1","addr":"172.21.15.196:6817","nonce":12772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":12772},{"type":"v1","addr":"172.21.15.196:6819","nonce":12772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":12772},{"type":"v1","addr":"172.21.15.196:6823","nonce":12772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":12772},{"type":"v1","addr":"172.21.15.196:6821","nonce":12772}]},"public_addr":"172.21.15.196:6817/12772","cluster_addr":"172.21.15.196:6819/12772","heartbeat_back_addr":"172.21.15.196:6823/12772","heartbeat_front_addr":"172.21.15.196:6821/12772","state":["exists","up"]},{"osd":2,"uuid":"ac1f2e05-9bb7-4768-b712-10e4d2ea4154","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":12773},{"type":"v1","addr":"172.21.15.196:6809","nonce":12773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":12773},{"type":"v1","addr":"172.21.15.196:6811","nonce":12773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":12773},{"type":"v1","addr":"172.21.15.196:6815","nonce":12773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":12773},{"type":"v1","addr":"172.21.15.196:6813","nonce":12773}]},"public_addr":"172.21.15.196:6809/12773","cluster_addr":"172.21.15.196:6811/12773","heartbeat_back_addr":"172.21.15.196:6815/12773","heartbeat_front_addr":"172.21.15.196:6813/12773","state":["exists","up"]},{"osd":3,"uuid":"bae5ab40-235b-4e0f-8d0d-524afd099b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":12775},{"type":"v1","addr":"172.21.15.196:6825","nonce":12775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":12775},{"type":"v1","addr":"172.21.15.196:6827","nonce":12775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":12775},{"type":"v1","addr":"172.21.15.196:6831","nonce":12775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":12775},{"type":"v1","addr":"172.21.15.196:6829","nonce":12775}]},"public_addr":"172.21.15.196:6825/12775","cluster_addr":"172.21.15.196:6827/12775","heartbeat_back_addr":"172.21.15.196:6831/12775","heartbeat_front_addr":"172.21.15.196:6829/12775","state":["exists","up"]},{"osd":4,"uuid":"275fc52c-58e4-4f35-a0cb-e117fe36bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":13040},{"type":"v1","addr":"172.21.15.80:6819","nonce":13040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6821","nonce":13040},{"type":"v1","addr":"172.21.15.80:6823","nonce":13040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6829","nonce":13040},{"type":"v1","addr":"172.21.15.80:6831","nonce":13040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":13040},{"type":"v1","addr":"172.21.15.80:6827","nonce":13040}]},"public_addr":"172.21.15.80:6819/13040","cluster_addr":"172.21.15.80:6823/13040","heartbeat_back_addr":"172.21.15.80:6831/13040","heartbeat_front_addr":"172.21.15.80:6827/13040","state":["exists","up"]},{"osd":5,"uuid":"9795b89c-e2f8-4abf-b9c7-796727a57002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":13037},{"type":"v1","addr":"172.21.15.80:6818","nonce":13037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":13037},{"type":"v1","addr":"172.21.15.80:6822","nonce":13037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":13037},{"type":"v1","addr":"172.21.15.80:6830","nonce":13037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":13037},{"type":"v1","addr":"172.21.15.80:6826","nonce":13037}]},"public_addr":"172.21.15.80:6818/13037","cluster_addr":"172.21.15.80:6822/13037","heartbeat_back_addr":"172.21.15.80:6830/13037","heartbeat_front_addr":"172.21.15.80:6826/13037","state":["exists","up"]},{"osd":6,"uuid":"e42df2e8-9cc5-4c7e-a243-bd18ccfbd002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":13039},{"type":"v1","addr":"172.21.15.80:6804","nonce":13039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":13039},{"type":"v1","addr":"172.21.15.80:6808","nonce":13039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":13039},{"type":"v1","addr":"172.21.15.80:6813","nonce":13039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":13039},{"type":"v1","addr":"172.21.15.80:6810","nonce":13039}]},"public_addr":"172.21.15.80:6804/13039","cluster_addr":"172.21.15.80:6808/13039","heartbeat_back_addr":"172.21.15.80:6813/13039","heartbeat_front_addr":"172.21.15.80:6810/13039","state":["exists","up"]},{"osd":7,"uuid":"ebd6c938-beff-4668-b70e-076ed22bb8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":13038},{"type":"v1","addr":"172.21.15.80:6801","nonce":13038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":13038},{"type":"v1","addr":"172.21.15.80:6805","nonce":13038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":13038},{"type":"v1","addr":"172.21.15.80:6817","nonce":13038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":13038},{"type":"v1","addr":"172.21.15.80:6811","nonce":13038}]},"public_addr":"172.21.15.80:6801/13038","cluster_addr":"172.21.15.80:6805/13038","heartbeat_back_addr":"172.21.15.80:6817/13038","heartbeat_front_addr":"172.21.15.80:6811/13038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.134515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.439836+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.476825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.176642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.452301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.427183+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.230250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.210168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-18T21:40:36.678 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.679+0000 7f5a225cd700 1 -- 172.21.15.196:0/25111458 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f59f804feb0 msgr2=0x7f59f8052370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.678 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.679+0000 7f5a225cd700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f59f804feb0 0x7f59f8052370 secure :-1 s=READY pgs=48 cs=0 l=1 rx=0x7f5a10008950 tx=0x7f5a10007a80).stop 2020-05-18T21:40:36.679 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.679+0000 7f5a225cd700 1 -- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 msgr2=0x7f5a1c13c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.679 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.679+0000 7f5a225cd700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 secure :-1 s=READY pgs=23 cs=0 l=1 rx=0x7f5a1c060960 tx=0x7f5a18001e70).stop 2020-05-18T21:40:36.679 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.679+0000 7f5a225cd700 1 -- 172.21.15.196:0/25111458 shutdown_connections 2020-05-18T21:40:36.679 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.679+0000 7f5a225cd700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f59f804feb0 0x7f59f8052370 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.679 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.679+0000 7f5a225cd700 1 --2- 172.21.15.196:0/25111458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a1c066910 0x7f5a1c13c470 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.680 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.679+0000 7f5a225cd700 1 -- 172.21.15.196:0/25111458 >> 172.21.15.196:0/25111458 conn(0x7f5a1c0ae900 msgr2=0x7f5a1c0ac730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:36.680 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.679+0000 7f5a225cd700 1 -- 172.21.15.196:0/25111458 shutdown_connections 2020-05-18T21:40:36.680 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.679+0000 7f5a225cd700 1 -- 172.21.15.196:0/25111458 wait complete. 2020-05-18T21:40:36.689 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 flush_pg_stats 2020-05-18T21:40:36.795 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.795+0000 7f83f3268700 1 Processor -- start 2020-05-18T21:40:36.796 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.795+0000 7f83f3268700 1 -- start start 2020-05-18T21:40:36.797 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.795+0000 7f83f3268700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f83ec05b210 0x7f83ec0a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:36.797 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.795+0000 7f83f3268700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f83ec0a7240 0x7f83ec1471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:36.797 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.795+0000 7f83f3268700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec0a7750 con 0x7f83ec0a66a0 2020-05-18T21:40:36.797 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.795+0000 7f83f3268700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec0a4e10 con 0x7f83ec0a7240 2020-05-18T21:40:36.798 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.795+0000 7f83f3268700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec0a7980 con 0x7f83ec05b210 2020-05-18T21:40:36.798 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.795+0000 7f83f1a65700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f83ec0a66a0 0x7f83ec0a6b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43254/0 (socket says 172.21.15.196:43254) 2020-05-18T21:40:36.798 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.795+0000 7f83f2266700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f83ec05b210 0x7f83ec0a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.798 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.795+0000 7f83f1a65700 1 -- 172.21.15.196:0/4166198762 learned_addr learned my addr 172.21.15.196:0/4166198762 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:36.798 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.795+0000 7f83f2a67700 1 --2- 172.21.15.196:0/4166198762 >> v2:172.21.15.80:3300/0 conn(0x7f83ec0a7240 0x7f83ec1471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.800 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f2266700 1 -- 172.21.15.196:0/4166198762 >> v2:172.21.15.80:3300/0 conn(0x7f83ec0a7240 msgr2=0x7f83ec1471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:36.800 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f2266700 1 --2- 172.21.15.196:0/4166198762 >> v2:172.21.15.80:3300/0 conn(0x7f83ec0a7240 0x7f83ec1471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:36.800 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f2266700 1 -- 172.21.15.196:0/4166198762 >> v1:172.21.15.196:6789/0 conn(0x7f83ec0a66a0 legacy=0x7f83ec0a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.801 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f2266700 1 -- 172.21.15.196:0/4166198762 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0a7c20 con 0x7f83ec05b210 2020-05-18T21:40:36.801 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f2266700 1 --2- 172.21.15.196:0/4166198762 >> v2:172.21.15.196:3300/0 conn(0x7f83ec05b210 0x7f83ec0a33a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f83e0002f70 tx=0x7f83e0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.801 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f2266700 1 --2- 172.21.15.196:0/4166198762 >> v2:172.21.15.196:3300/0 conn(0x7f83ec05b210 0x7f83ec0a33a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f83e0002f70 tx=0x7f83e0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.801 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f2266700 1 --2- 172.21.15.196:0/4166198762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec05b210 0x7f83ec0a33a0 secure :-1 s=READY pgs=131 cs=0 l=1 rx=0x7f83e0002f70 tx=0x7f83e0007a80).ready entity=mon.1 client_cookie=8870825b033c96cd server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.801 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f2266700 1 --2- 172.21.15.196:0/4166198762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec05b210 0x7f83ec0a33a0 secure :-1 s=THROTTLE_DONE pgs=131 cs=0 l=1 rx=0x7f83e0002f70 tx=0x7f83e0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.802 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f2266700 1 --2- 172.21.15.196:0/4166198762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec05b210 0x7f83ec0a33a0 secure :-1 s=THROTTLE_DONE pgs=131 cs=0 l=1 rx=0x7f83e0002f70 tx=0x7f83e0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.802 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f1264700 1 -- 172.21.15.196:0/4166198762 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83e0008e50 con 0x7f83ec05b210 2020-05-18T21:40:36.802 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f2266700 1 --2- 172.21.15.196:0/4166198762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec05b210 0x7f83ec0a33a0 secure :-1 s=THROTTLE_DONE pgs=131 cs=0 l=1 rx=0x7f83e0002f70 tx=0x7f83e0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.802 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f1264700 1 -- 172.21.15.196:0/4166198762 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e00115f0 con 0x7f83ec05b210 2020-05-18T21:40:36.802 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f1264700 1 -- 172.21.15.196:0/4166198762 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83e0011a10 con 0x7f83ec05b210 2020-05-18T21:40:36.803 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f3268700 1 -- 172.21.15.196:0/4166198762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec05b210 msgr2=0x7f83ec0a33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.803 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f3268700 1 --2- 172.21.15.196:0/4166198762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec05b210 0x7f83ec0a33a0 secure :-1 s=READY pgs=131 cs=0 l=1 rx=0x7f83e0002f70 tx=0x7f83e0007a80).stop 2020-05-18T21:40:36.803 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f3268700 1 -- 172.21.15.196:0/4166198762 shutdown_connections 2020-05-18T21:40:36.803 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f3268700 1 --2- 172.21.15.196:0/4166198762 >> v2:172.21.15.80:3300/0 conn(0x7f83ec0a7240 0x7f83ec1471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:36.803 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f3268700 1 --2- 172.21.15.196:0/4166198762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec05b210 0x7f83ec0a33a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.804 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f3268700 1 -- 172.21.15.196:0/4166198762 >> 172.21.15.196:0/4166198762 conn(0x7f83ec060640 msgr2=0x7f83ec05a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:36.804 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f3268700 1 -- 172.21.15.196:0/4166198762 shutdown_connections 2020-05-18T21:40:36.804 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f3268700 1 -- 172.21.15.196:0/4166198762 wait complete. 2020-05-18T21:40:36.808 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f3268700 1 Processor -- start 2020-05-18T21:40:36.808 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.799+0000 7f83f3268700 1 -- start start 2020-05-18T21:40:36.808 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f3268700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f83ec05b210 0x7f83ec065860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.808 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f3268700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f83ec0a66a0 0x7f83ec0a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.808 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f3268700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec0642c0 con 0x7f83ec0a7240 2020-05-18T21:40:36.809 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f3268700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec0644f0 con 0x7f83ec0a66a0 2020-05-18T21:40:36.809 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f3268700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec064790 con 0x7f83ec05b210 2020-05-18T21:40:36.809 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f1a65700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f83ec0a66a0 0x7f83ec0a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.809 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f2a67700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f83ec0a7240 0x7f83ec13f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43264/0 (socket says 172.21.15.196:43264) 2020-05-18T21:40:36.809 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f2a67700 1 -- 172.21.15.196:0/2911830247 learned_addr learned my addr 172.21.15.196:0/2911830247 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:36.810 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f2266700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f83ec05b210 0x7f83ec065860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.810 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83eb7fe700 1 -- 172.21.15.196:0/2911830247 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190294799 0 0) 0x7f83ec0642c0 con 0x7f83ec0a7240 2020-05-18T21:40:36.810 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83eb7fe700 1 -- 172.21.15.196:0/2911830247 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83cc002fe0 con 0x7f83ec0a7240 2020-05-18T21:40:36.810 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f1a65700 1 -- 172.21.15.196:0/2911830247 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f83ec05b210 msgr2=0x7f83ec065860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.810 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f1a65700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f83ec05b210 0x7f83ec065860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.811 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f1a65700 1 -- 172.21.15.196:0/2911830247 >> v1:172.21.15.196:6789/0 conn(0x7f83ec0a7240 legacy=0x7f83ec13f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.811 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f1a65700 1 -- 172.21.15.196:0/2911830247 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec064a30 con 0x7f83ec0a66a0 2020-05-18T21:40:36.811 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f1a65700 1 --2- 172.21.15.196:0/2911830247 >> v2:172.21.15.196:3300/0 conn(0x7f83ec0a66a0 0x7f83ec0a0910 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f83ec0a22b0 tx=0x7f83e40108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.811 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f1a65700 1 --2- 172.21.15.196:0/2911830247 >> v2:172.21.15.196:3300/0 conn(0x7f83ec0a66a0 0x7f83ec0a0910 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f83ec0a22b0 tx=0x7f83e40108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.811 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f1a65700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0a66a0 0x7f83ec0a0910 secure :-1 s=READY pgs=133 cs=0 l=1 rx=0x7f83ec0a22b0 tx=0x7f83e40108d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.812 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f1a65700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0a66a0 0x7f83ec0a0910 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7f83ec0a22b0 tx=0x7f83e40108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.812 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83eb7fe700 1 -- 172.21.15.196:0/2911830247 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83e4005ce0 con 0x7f83ec0a66a0 2020-05-18T21:40:36.812 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f1a65700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0a66a0 0x7f83ec0a0910 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7f83ec0a22b0 tx=0x7f83e40108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.812 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83eb7fe700 1 -- 172.21.15.196:0/2911830247 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e4005e40 con 0x7f83ec0a66a0 2020-05-18T21:40:36.812 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f3268700 1 -- 172.21.15.196:0/2911830247 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec09b2b0 con 0x7f83ec0a66a0 2020-05-18T21:40:36.812 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f1a65700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0a66a0 0x7f83ec0a0910 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7f83ec0a22b0 tx=0x7f83e40108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.813 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83eb7fe700 1 -- 172.21.15.196:0/2911830247 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83e40062c0 con 0x7f83ec0a66a0 2020-05-18T21:40:36.813 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f3268700 1 -- 172.21.15.196:0/2911830247 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec065cc0 con 0x7f83ec0a66a0 2020-05-18T21:40:36.813 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f3268700 1 -- 172.21.15.196:0/2911830247 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83ec13fbb0 con 0x7f83ec0a66a0 2020-05-18T21:40:36.813 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.803+0000 7f83f1a65700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0a66a0 0x7f83ec0a0910 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7f83ec0a22b0 tx=0x7f83e40108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.813 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83eb7fe700 1 -- 172.21.15.196:0/2911830247 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f83e4006580 con 0x7f83ec0a66a0 2020-05-18T21:40:36.814 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83f1a65700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0a66a0 0x7f83ec0a0910 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7f83ec0a22b0 tx=0x7f83e40108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.814 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83f1a65700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0a66a0 0x7f83ec0a0910 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7f83ec0a22b0 tx=0x7f83e40108d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.814 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83eb7fe700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f83cc04fd80 0x7f83cc052240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.814 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83eb7fe700 1 -- 172.21.15.196:0/2911830247 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f83e40090c0 con 0x7f83ec0a66a0 2020-05-18T21:40:36.814 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83f2266700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f83cc04fd80 0x7f83cc052240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.815 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83eb7fe700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:6800/12771,v1:172.21.15.196:6801/12771] conn(0x7f83cc0568f0 0x7f83cc058d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.815 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83eb7fe700 1 -- 172.21.15.196:0/2911830247 --> [v2:172.21.15.196:6800/12771,v1:172.21.15.196:6801/12771] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f83cc0592d0 con 0x7f83cc0568f0 2020-05-18T21:40:36.815 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83eb7fe700 1 -- 172.21.15.196:0/2911830247 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83e4063fc0 con 0x7f83ec0a66a0 2020-05-18T21:40:36.815 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83f2266700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f83cc04fd80 0x7f83cc052240 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f83e0002f70 tx=0x7f83e0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.815 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83f2a67700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:6800/12771,v1:172.21.15.196:6801/12771] conn(0x7f83cc0568f0 0x7f83cc058d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.815 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83f2266700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f83cc04fd80 0x7f83cc052240 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f83e0002f70 tx=0x7f83e0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.816 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83f2266700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f83cc04fd80 0x7f83cc052240 secure :-1 s=READY pgs=49 cs=0 l=1 rx=0x7f83e0002f70 tx=0x7f83e0007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.816 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83f2a67700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:6800/12771,v1:172.21.15.196:6801/12771] conn(0x7f83cc0568f0 0x7f83cc058d10 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.816 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.807+0000 7f83eb7fe700 1 -- 172.21.15.196:0/2911830247 <== osd.0 v2:172.21.15.196:6800/12771 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23263 (crc 0 0 0) 0x7f83cc0592d0 con 0x7f83cc0568f0 2020-05-18T21:40:36.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83f3268700 1 -- 172.21.15.196:0/2911830247 --> [v2:172.21.15.196:6800/12771,v1:172.21.15.196:6801/12771] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f83ec0a0fb0 con 0x7f83cc0568f0 2020-05-18T21:40:36.826 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83eb7fe700 1 -- 172.21.15.196:0/2911830247 <== osd.0 v2:172.21.15.196:6800/12771 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f83ec0a0fb0 con 0x7f83cc0568f0 2020-05-18T21:40:36.827 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83f3268700 1 -- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:6800/12771,v1:172.21.15.196:6801/12771] conn(0x7f83cc0568f0 msgr2=0x7f83cc058d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83f3268700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:6800/12771,v1:172.21.15.196:6801/12771] conn(0x7f83cc0568f0 0x7f83cc058d10 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83f3268700 1 -- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f83cc04fd80 msgr2=0x7f83cc052240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83f3268700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f83cc04fd80 0x7f83cc052240 secure :-1 s=READY pgs=49 cs=0 l=1 rx=0x7f83e0002f70 tx=0x7f83e0007a80).stop 2020-05-18T21:40:36.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83f3268700 1 -- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0a66a0 msgr2=0x7f83ec0a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83f3268700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0a66a0 0x7f83ec0a0910 secure :-1 s=READY pgs=133 cs=0 l=1 rx=0x7f83ec0a22b0 tx=0x7f83e40108d0).stop 2020-05-18T21:40:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83f2a67700 1 -- 172.21.15.196:0/2911830247 reap_dead start 2020-05-18T21:40:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83f3268700 1 -- 172.21.15.196:0/2911830247 shutdown_connections 2020-05-18T21:40:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83f3268700 1 --2- 172.21.15.196:0/2911830247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0a66a0 0x7f83ec0a0910 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83f3268700 1 -- 172.21.15.196:0/2911830247 >> 172.21.15.196:0/2911830247 conn(0x7f83ec060640 msgr2=0x7f83ec05eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83f3268700 1 -- 172.21.15.196:0/2911830247 shutdown_connections 2020-05-18T21:40:36.830 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.827+0000 7f83f3268700 1 -- 172.21.15.196:0/2911830247 wait complete. 2020-05-18T21:40:36.838 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2020-05-18T21:40:36.838 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 flush_pg_stats 2020-05-18T21:40:36.947 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99ca3ac700 1 Processor -- start 2020-05-18T21:40:36.947 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99ca3ac700 1 -- start start 2020-05-18T21:40:36.950 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99ca3ac700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f99c40a91e0 0x7f99c413b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:36.950 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99ca3ac700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c405b670 con 0x7f99c405c1b0 2020-05-18T21:40:36.950 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99ca3ac700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c40a9760 con 0x7f99c4066680 2020-05-18T21:40:36.950 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99ca3ac700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c413b630 con 0x7f99c40a91e0 2020-05-18T21:40:36.950 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99c93aa700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f99c4066680 0x7f99c405baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43270/0 (socket says 172.21.15.196:43270) 2020-05-18T21:40:36.951 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99c93aa700 1 -- 172.21.15.196:0/2016259033 learned_addr learned my addr 172.21.15.196:0/2016259033 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:36.951 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99c9bab700 1 --2- 172.21.15.196:0/2016259033 >> v2:172.21.15.80:3300/0 conn(0x7f99c40a91e0 0x7f99c413b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.951 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99b3fff700 1 -- 172.21.15.196:0/2016259033 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287067932 0 0) 0x7f99c40a9760 con 0x7f99c4066680 2020-05-18T21:40:36.951 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99b3fff700 1 -- 172.21.15.196:0/2016259033 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99ac003270 con 0x7f99c4066680 2020-05-18T21:40:36.951 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99c9bab700 1 -- 172.21.15.196:0/2016259033 >> v1:172.21.15.196:6789/0 conn(0x7f99c4066680 legacy=0x7f99c405baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.952 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99c9bab700 1 -- 172.21.15.196:0/2016259033 >> v1:172.21.15.80:6789/0 conn(0x7f99c405c1b0 legacy=0x7f99c413aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.952 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99c9bab700 1 -- 172.21.15.196:0/2016259033 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c40a9a00 con 0x7f99c40a91e0 2020-05-18T21:40:36.952 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99c9bab700 1 --2- 172.21.15.196:0/2016259033 >> v2:172.21.15.80:3300/0 conn(0x7f99c40a91e0 0x7f99c413b1d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.952 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99c9bab700 1 --2- 172.21.15.196:0/2016259033 >> v2:172.21.15.80:3300/0 conn(0x7f99c40a91e0 0x7f99c413b1d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.952 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99c9bab700 1 --2- 172.21.15.196:0/2016259033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c413b1d0 secure :-1 s=READY pgs=96 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).ready entity=mon.0 client_cookie=21d7f04f486489d9 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.953 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99c9bab700 1 --2- 172.21.15.196:0/2016259033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c413b1d0 secure :-1 s=THROTTLE_DONE pgs=96 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.953 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99b3fff700 1 -- 172.21.15.196:0/2016259033 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99bc011e60 con 0x7f99c40a91e0 2020-05-18T21:40:36.953 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99c9bab700 1 --2- 172.21.15.196:0/2016259033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c413b1d0 secure :-1 s=THROTTLE_DONE pgs=96 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.953 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99b3fff700 1 -- 172.21.15.196:0/2016259033 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99bc00a180 con 0x7f99c40a91e0 2020-05-18T21:40:36.953 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99c9bab700 1 --2- 172.21.15.196:0/2016259033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c413b1d0 secure :-1 s=THROTTLE_DONE pgs=96 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.953 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.947+0000 7f99b3fff700 1 -- 172.21.15.196:0/2016259033 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99bc00a4e0 con 0x7f99c40a91e0 2020-05-18T21:40:36.954 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 -- 172.21.15.196:0/2016259033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 msgr2=0x7f99c413b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.954 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 --2- 172.21.15.196:0/2016259033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c413b1d0 secure :-1 s=READY pgs=96 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).stop 2020-05-18T21:40:36.954 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 -- 172.21.15.196:0/2016259033 shutdown_connections 2020-05-18T21:40:36.954 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 --2- 172.21.15.196:0/2016259033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c413b1d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.954 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 -- 172.21.15.196:0/2016259033 >> 172.21.15.196:0/2016259033 conn(0x7f99c4069ac0 msgr2=0x7f99c4065c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:36.955 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 -- 172.21.15.196:0/2016259033 shutdown_connections 2020-05-18T21:40:36.955 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 -- 172.21.15.196:0/2016259033 wait complete. 2020-05-18T21:40:36.958 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 Processor -- start 2020-05-18T21:40:36.959 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 -- start start 2020-05-18T21:40:36.959 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f99c405c1b0 0x7f99c4137290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.959 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f99c40a91e0 0x7f99c40a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.959 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c40a53f0 con 0x7f99c4066680 2020-05-18T21:40:36.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c40a5630 con 0x7f99c40a91e0 2020-05-18T21:40:36.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c40af080 con 0x7f99c405c1b0 2020-05-18T21:40:36.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c8ba9700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f99c4066680 0x7f99c4133de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43274/0 (socket says 172.21.15.196:43274) 2020-05-18T21:40:36.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c8ba9700 1 -- 172.21.15.196:0/1508731633 learned_addr learned my addr 172.21.15.196:0/1508731633 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:36.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c93aa700 1 --2- 172.21.15.196:0/1508731633 >> v2:172.21.15.196:3300/0 conn(0x7f99c405c1b0 0x7f99c4137290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c9bab700 1 --2- 172.21.15.196:0/1508731633 >> v2:172.21.15.80:3300/0 conn(0x7f99c40a91e0 0x7f99c40a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99b27fc700 1 -- 172.21.15.196:0/1508731633 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715243152 0 0) 0x7f99c405b670 con 0x7f99c4066680 2020-05-18T21:40:36.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99b27fc700 1 -- 172.21.15.196:0/1508731633 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99a4002fe0 con 0x7f99c4066680 2020-05-18T21:40:36.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c9bab700 1 -- 172.21.15.196:0/1508731633 >> v2:172.21.15.196:3300/0 conn(0x7f99c405c1b0 msgr2=0x7f99c4137290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c9bab700 1 --2- 172.21.15.196:0/1508731633 >> v2:172.21.15.196:3300/0 conn(0x7f99c405c1b0 0x7f99c4137290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c9bab700 1 -- 172.21.15.196:0/1508731633 >> v1:172.21.15.196:6789/0 conn(0x7f99c4066680 legacy=0x7f99c4133de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.962 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c9bab700 1 -- 172.21.15.196:0/1508731633 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c41344f0 con 0x7f99c40a91e0 2020-05-18T21:40:36.962 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c9bab700 1 --2- 172.21.15.196:0/1508731633 >> v2:172.21.15.80:3300/0 conn(0x7f99c40a91e0 0x7f99c40a4f90 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.962 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c9bab700 1 --2- 172.21.15.196:0/1508731633 >> v2:172.21.15.80:3300/0 conn(0x7f99c40a91e0 0x7f99c40a4f90 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.962 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c9bab700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c40a4f90 secure :-1 s=READY pgs=97 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.962 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c9bab700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c40a4f90 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99b27fc700 1 -- 172.21.15.196:0/1508731633 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99bc00ae50 con 0x7f99c40a91e0 2020-05-18T21:40:36.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c9bab700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c40a4f90 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99b27fc700 1 -- 172.21.15.196:0/1508731633 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99bc005c70 con 0x7f99c40a91e0 2020-05-18T21:40:36.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99ca3ac700 1 -- 172.21.15.196:0/1508731633 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c40af520 con 0x7f99c40a91e0 2020-05-18T21:40:36.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99c9bab700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c40a4f90 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.951+0000 7f99b27fc700 1 -- 172.21.15.196:0/1508731633 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99bc007170 con 0x7f99c40a91e0 2020-05-18T21:40:36.964 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99ca3ac700 1 -- 172.21.15.196:0/1508731633 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c40ad8a0 con 0x7f99c40a91e0 2020-05-18T21:40:36.964 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99ca3ac700 1 -- 172.21.15.196:0/1508731633 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f99c40569e0 con 0x7f99c40a91e0 2020-05-18T21:40:36.964 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99c9bab700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c40a4f90 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.964 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99b27fc700 1 -- 172.21.15.196:0/1508731633 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f99bc007320 con 0x7f99c40a91e0 2020-05-18T21:40:36.964 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99c9bab700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c40a4f90 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.965 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99c9bab700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c40a4f90 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.965 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99b27fc700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f99a404fd40 0x7f99a4052200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.965 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99b27fc700 1 -- 172.21.15.196:0/1508731633 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f99bc064c80 con 0x7f99c40a91e0 2020-05-18T21:40:36.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99c93aa700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f99a404fd40 0x7f99a4052200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99b27fc700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.196:6816/12772,v1:172.21.15.196:6817/12772] conn(0x7f99a4056820 0x7f99a4058c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:36.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99b27fc700 1 -- 172.21.15.196:0/1508731633 --> [v2:172.21.15.196:6816/12772,v1:172.21.15.196:6817/12772] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99a4059200 con 0x7f99a4056820 2020-05-18T21:40:36.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99b27fc700 1 -- 172.21.15.196:0/1508731633 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99bc065060 con 0x7f99c40a91e0 2020-05-18T21:40:36.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99c93aa700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f99a404fd40 0x7f99a4052200 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f99b4004880 tx=0x7f99c405e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99c93aa700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f99a404fd40 0x7f99a4052200 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f99b4004880 tx=0x7f99c405e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:36.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99c8ba9700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.196:6816/12772,v1:172.21.15.196:6817/12772] conn(0x7f99a4056820 0x7f99a4058c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:36.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99c93aa700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f99a404fd40 0x7f99a4052200 secure :-1 s=READY pgs=50 cs=0 l=1 rx=0x7f99b4004880 tx=0x7f99c405e380).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.955+0000 7f99c8ba9700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.196:6816/12772,v1:172.21.15.196:6817/12772] conn(0x7f99a4056820 0x7f99a4058c40 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:36.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.959+0000 7f99b27fc700 1 -- 172.21.15.196:0/1508731633 <== osd.1 v2:172.21.15.196:6816/12772 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23263 (crc 0 0 0) 0x7f99a4059200 con 0x7f99a4056820 2020-05-18T21:40:36.974 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99ca3ac700 1 -- 172.21.15.196:0/1508731633 --> [v2:172.21.15.196:6816/12772,v1:172.21.15.196:6817/12772] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f99c406a830 con 0x7f99a4056820 2020-05-18T21:40:36.974 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99b27fc700 1 -- 172.21.15.196:0/1508731633 <== osd.1 v2:172.21.15.196:6816/12772 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f99c406a830 con 0x7f99a4056820 2020-05-18T21:40:36.976 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99ca3ac700 1 -- 172.21.15.196:0/1508731633 >> [v2:172.21.15.196:6816/12772,v1:172.21.15.196:6817/12772] conn(0x7f99a4056820 msgr2=0x7f99a4058c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.976 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99ca3ac700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.196:6816/12772,v1:172.21.15.196:6817/12772] conn(0x7f99a4056820 0x7f99a4058c40 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.976 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99ca3ac700 1 -- 172.21.15.196:0/1508731633 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f99a404fd40 msgr2=0x7f99a4052200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.976 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99ca3ac700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f99a404fd40 0x7f99a4052200 secure :-1 s=READY pgs=50 cs=0 l=1 rx=0x7f99b4004880 tx=0x7f99c405e380).stop 2020-05-18T21:40:36.977 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99ca3ac700 1 -- 172.21.15.196:0/1508731633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 msgr2=0x7f99c40a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:36.977 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99ca3ac700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c40a4f90 secure :-1 s=READY pgs=97 cs=0 l=1 rx=0x7f99bc0032f0 tx=0x7f99bc007e00).stop 2020-05-18T21:40:36.977 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99c9bab700 1 -- 172.21.15.196:0/1508731633 reap_dead start 2020-05-18T21:40:36.977 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99ca3ac700 1 -- 172.21.15.196:0/1508731633 shutdown_connections 2020-05-18T21:40:36.977 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99ca3ac700 1 --2- 172.21.15.196:0/1508731633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c40a91e0 0x7f99c40a4f90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:36.978 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99ca3ac700 1 -- 172.21.15.196:0/1508731633 >> 172.21.15.196:0/1508731633 conn(0x7f99c4069ac0 msgr2=0x7f99c405eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:36.978 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99ca3ac700 1 -- 172.21.15.196:0/1508731633 shutdown_connections 2020-05-18T21:40:36.978 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:36.975+0000 7f99ca3ac700 1 -- 172.21.15.196:0/1508731633 wait complete. 2020-05-18T21:40:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2020-05-18T21:40:36.989 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 flush_pg_stats 2020-05-18T21:40:37.098 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa5ca08700 1 Processor -- start 2020-05-18T21:40:37.098 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa5ca08700 1 -- start start 2020-05-18T21:40:37.099 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa5ca08700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffa5805b210 0x7ffa580a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.099 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa5ca08700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa580a66a0 0x7ffa580a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.099 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa5ca08700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa580a7750 con 0x7ffa580a6f90 2020-05-18T21:40:37.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa5ca08700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa580a4e10 con 0x7ffa5805b210 2020-05-18T21:40:37.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa5ca08700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa580a7980 con 0x7ffa580a66a0 2020-05-18T21:40:37.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa577fe700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffa5805b210 0x7ffa580a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa4effd700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa580a66a0 0x7ffa580a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa577fe700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffa5805b210 0x7ffa580a33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.196:38750/0 (socket says 172.21.15.196:38750) 2020-05-18T21:40:37.101 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa577fe700 1 -- 172.21.15.196:0/157026893 learned_addr learned my addr 172.21.15.196:0/157026893 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:37.102 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa56ffd700 1 -- 172.21.15.196:0/157026893 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465638579 0 0) 0x7ffa580a7750 con 0x7ffa580a6f90 2020-05-18T21:40:37.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa56ffd700 1 -- 172.21.15.196:0/157026893 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa38003270 con 0x7ffa580a6f90 2020-05-18T21:40:37.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa577fe700 1 -- 172.21.15.196:0/157026893 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa580a66a0 msgr2=0x7ffa580a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:37.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa577fe700 1 --2- 172.21.15.196:0/157026893 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa580a66a0 0x7ffa580a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa577fe700 1 -- 172.21.15.196:0/157026893 >> v1:172.21.15.80:6789/0 conn(0x7ffa580a6f90 legacy=0x7ffa5813fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa577fe700 1 -- 172.21.15.196:0/157026893 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa580a7c20 con 0x7ffa5805b210 2020-05-18T21:40:37.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa4effd700 1 --2- 172.21.15.196:0/157026893 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa580a66a0 0x7ffa580a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:37.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa577fe700 1 --2- 172.21.15.196:0/157026893 >> v2:172.21.15.80:3300/0 conn(0x7ffa5805b210 0x7ffa580a33a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7ffa48002f70 tx=0x7ffa48007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa577fe700 1 --2- 172.21.15.196:0/157026893 >> v2:172.21.15.80:3300/0 conn(0x7ffa5805b210 0x7ffa580a33a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7ffa48002f70 tx=0x7ffa48007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa577fe700 1 --2- 172.21.15.196:0/157026893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa5805b210 0x7ffa580a33a0 secure :-1 s=READY pgs=99 cs=0 l=1 rx=0x7ffa48002f70 tx=0x7ffa48007a80).ready entity=mon.0 client_cookie=4c06895ad22e99e3 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa577fe700 1 --2- 172.21.15.196:0/157026893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa5805b210 0x7ffa580a33a0 secure :-1 s=THROTTLE_DONE pgs=99 cs=0 l=1 rx=0x7ffa48002f70 tx=0x7ffa48007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.105 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa56ffd700 1 -- 172.21.15.196:0/157026893 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa48011600 con 0x7ffa5805b210 2020-05-18T21:40:37.105 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa577fe700 1 --2- 172.21.15.196:0/157026893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa5805b210 0x7ffa580a33a0 secure :-1 s=THROTTLE_DONE pgs=99 cs=0 l=1 rx=0x7ffa48002f70 tx=0x7ffa48007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.105 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa577fe700 1 --2- 172.21.15.196:0/157026893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa5805b210 0x7ffa580a33a0 secure :-1 s=THROTTLE_DONE pgs=99 cs=0 l=1 rx=0x7ffa48002f70 tx=0x7ffa48007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.105 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa56ffd700 1 -- 172.21.15.196:0/157026893 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa48011760 con 0x7ffa5805b210 2020-05-18T21:40:37.105 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa56ffd700 1 -- 172.21.15.196:0/157026893 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa48011b80 con 0x7ffa5805b210 2020-05-18T21:40:37.106 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa5ca08700 1 -- 172.21.15.196:0/157026893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa5805b210 msgr2=0x7ffa580a33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.106 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.099+0000 7ffa5ca08700 1 --2- 172.21.15.196:0/157026893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa5805b210 0x7ffa580a33a0 secure :-1 s=READY pgs=99 cs=0 l=1 rx=0x7ffa48002f70 tx=0x7ffa48007a80).stop 2020-05-18T21:40:37.106 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 -- 172.21.15.196:0/157026893 shutdown_connections 2020-05-18T21:40:37.106 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 --2- 172.21.15.196:0/157026893 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa580a66a0 0x7ffa580a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.106 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 --2- 172.21.15.196:0/157026893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa5805b210 0x7ffa580a33a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.106 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 -- 172.21.15.196:0/157026893 >> 172.21.15.196:0/157026893 conn(0x7ffa58060640 msgr2=0x7ffa5805a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:37.107 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 -- 172.21.15.196:0/157026893 shutdown_connections 2020-05-18T21:40:37.107 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 -- 172.21.15.196:0/157026893 wait complete. 2020-05-18T21:40:37.118 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 Processor -- start 2020-05-18T21:40:37.118 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 -- start start 2020-05-18T21:40:37.118 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffa5805b210 0x7ffa5809c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.118 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffa580a66a0 0x7ffa5809b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.118 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa580a6f90 0x7ffa5814a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.119 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa58149970 con 0x7ffa5805b210 2020-05-18T21:40:37.119 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5809be90 con 0x7ffa580a66a0 2020-05-18T21:40:37.119 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa5ca08700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58149bd0 con 0x7ffa580a6f90 2020-05-18T21:40:37.119 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa4effd700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffa580a66a0 0x7ffa5809b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.119 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa4effd700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffa580a66a0 0x7ffa5809b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34530/0 (socket says 172.21.15.196:34530) 2020-05-18T21:40:37.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa577fe700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffa5805b210 0x7ffa5809c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa4effd700 1 -- 172.21.15.196:0/2792852730 learned_addr learned my addr 172.21.15.196:0/2792852730 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:37.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.103+0000 7ffa57fff700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa580a6f90 0x7ffa5814a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 -- 172.21.15.196:0/2792852730 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa580a6f90 msgr2=0x7ffa5814a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.120 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa580a6f90 0x7ffa5814a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 -- 172.21.15.196:0/2792852730 >> v2:172.21.15.80:3300/0 conn(0x7ffa5805b210 msgr2=0x7ffa5809c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 --2- 172.21.15.196:0/2792852730 >> v2:172.21.15.80:3300/0 conn(0x7ffa5805b210 0x7ffa5809c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 -- 172.21.15.196:0/2792852730 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa58098770 con 0x7ffa580a66a0 2020-05-18T21:40:37.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 --2- 172.21.15.196:0/2792852730 >> v2:172.21.15.196:3300/0 conn(0x7ffa580a66a0 0x7ffa5809b5f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ffa40007960 tx=0x7ffa40010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 --2- 172.21.15.196:0/2792852730 >> v2:172.21.15.196:3300/0 conn(0x7ffa580a66a0 0x7ffa5809b5f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ffa40007960 tx=0x7ffa40010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.121 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa580a66a0 0x7ffa5809b5f0 secure :-1 s=READY pgs=136 cs=0 l=1 rx=0x7ffa40007960 tx=0x7ffa40010f00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.122 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa580a66a0 0x7ffa5809b5f0 secure :-1 s=THROTTLE_DONE pgs=136 cs=0 l=1 rx=0x7ffa40007960 tx=0x7ffa40010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.122 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa557fa700 1 -- 172.21.15.196:0/2792852730 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa40012b50 con 0x7ffa580a66a0 2020-05-18T21:40:37.122 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa580a66a0 0x7ffa5809b5f0 secure :-1 s=THROTTLE_DONE pgs=136 cs=0 l=1 rx=0x7ffa40007960 tx=0x7ffa40010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.122 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa580a66a0 0x7ffa5809b5f0 secure :-1 s=THROTTLE_DONE pgs=136 cs=0 l=1 rx=0x7ffa40007960 tx=0x7ffa40010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.122 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa557fa700 1 -- 172.21.15.196:0/2792852730 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa40012cb0 con 0x7ffa580a66a0 2020-05-18T21:40:37.123 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa5ca08700 1 -- 172.21.15.196:0/2792852730 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5809cb10 con 0x7ffa580a66a0 2020-05-18T21:40:37.123 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa557fa700 1 -- 172.21.15.196:0/2792852730 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa4000c740 con 0x7ffa580a66a0 2020-05-18T21:40:37.123 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa5ca08700 1 -- 172.21.15.196:0/2792852730 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5809bbf0 con 0x7ffa580a66a0 2020-05-18T21:40:37.123 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa580a66a0 0x7ffa5809b5f0 secure :-1 s=THROTTLE_DONE pgs=136 cs=0 l=1 rx=0x7ffa40007960 tx=0x7ffa40010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.123 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa5ca08700 1 -- 172.21.15.196:0/2792852730 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa58149dd0 con 0x7ffa580a66a0 2020-05-18T21:40:37.124 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa557fa700 1 -- 172.21.15.196:0/2792852730 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7ffa4000e1a0 con 0x7ffa580a66a0 2020-05-18T21:40:37.124 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa580a66a0 0x7ffa5809b5f0 secure :-1 s=THROTTLE_DONE pgs=136 cs=0 l=1 rx=0x7ffa40007960 tx=0x7ffa40010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.124 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.107+0000 7ffa4effd700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa580a66a0 0x7ffa5809b5f0 secure :-1 s=THROTTLE_DONE pgs=136 cs=0 l=1 rx=0x7ffa40007960 tx=0x7ffa40010f00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.124 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.111+0000 7ffa557fa700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ffa3c04fa20 0x7ffa3c051ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.124 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.111+0000 7ffa557fa700 1 -- 172.21.15.196:0/2792852730 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffa400640c0 con 0x7ffa580a66a0 2020-05-18T21:40:37.124 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.111+0000 7ffa577fe700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ffa3c04fa20 0x7ffa3c051ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.125 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.111+0000 7ffa557fa700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:6808/12773,v1:172.21.15.196:6809/12773] conn(0x7ffa3c056680 0x7ffa3c058aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.125 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.111+0000 7ffa557fa700 1 -- 172.21.15.196:0/2792852730 --> [v2:172.21.15.196:6808/12773,v1:172.21.15.196:6809/12773] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa3c059060 con 0x7ffa3c056680 2020-05-18T21:40:37.125 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.111+0000 7ffa557fa700 1 -- 172.21.15.196:0/2792852730 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa40065460 con 0x7ffa580a66a0 2020-05-18T21:40:37.125 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.111+0000 7ffa577fe700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ffa3c04fa20 0x7ffa3c051ee0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ffa48002f70 tx=0x7ffa48007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.125 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.111+0000 7ffa57fff700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:6808/12773,v1:172.21.15.196:6809/12773] conn(0x7ffa3c056680 0x7ffa3c058aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.126 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.111+0000 7ffa577fe700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ffa3c04fa20 0x7ffa3c051ee0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ffa48002f70 tx=0x7ffa48007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.126 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.111+0000 7ffa577fe700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ffa3c04fa20 0x7ffa3c051ee0 secure :-1 s=READY pgs=51 cs=0 l=1 rx=0x7ffa48002f70 tx=0x7ffa48007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.126 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.111+0000 7ffa57fff700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:6808/12773,v1:172.21.15.196:6809/12773] conn(0x7ffa3c056680 0x7ffa3c058aa0 crc :-1 s=READY pgs=6 cs=0 l=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.126 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.111+0000 7ffa557fa700 1 -- 172.21.15.196:0/2792852730 <== osd.2 v2:172.21.15.196:6808/12773 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23263 (crc 0 0 0) 0x7ffa3c059060 con 0x7ffa3c056680 2020-05-18T21:40:37.131 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.131+0000 7ffa5ca08700 1 -- 172.21.15.196:0/2792852730 --> [v2:172.21.15.196:6808/12773,v1:172.21.15.196:6809/12773] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa580a8520 con 0x7ffa3c056680 2020-05-18T21:40:37.133 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.131+0000 7ffa557fa700 1 -- 172.21.15.196:0/2792852730 <== osd.2 v2:172.21.15.196:6808/12773 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffa580a8520 con 0x7ffa3c056680 2020-05-18T21:40:37.134 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.135+0000 7ffa5ca08700 1 -- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:6808/12773,v1:172.21.15.196:6809/12773] conn(0x7ffa3c056680 msgr2=0x7ffa3c058aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.134 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.135+0000 7ffa5ca08700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:6808/12773,v1:172.21.15.196:6809/12773] conn(0x7ffa3c056680 0x7ffa3c058aa0 crc :-1 s=READY pgs=6 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.134 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.135+0000 7ffa5ca08700 1 -- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ffa3c04fa20 msgr2=0x7ffa3c051ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.135 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.135+0000 7ffa5ca08700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ffa3c04fa20 0x7ffa3c051ee0 secure :-1 s=READY pgs=51 cs=0 l=1 rx=0x7ffa48002f70 tx=0x7ffa48007a80).stop 2020-05-18T21:40:37.135 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.135+0000 7ffa5ca08700 1 -- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa580a66a0 msgr2=0x7ffa5809b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.135 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.135+0000 7ffa5ca08700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa580a66a0 0x7ffa5809b5f0 secure :-1 s=READY pgs=136 cs=0 l=1 rx=0x7ffa40007960 tx=0x7ffa40010f00).stop 2020-05-18T21:40:37.135 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.135+0000 7ffa57fff700 1 -- 172.21.15.196:0/2792852730 reap_dead start 2020-05-18T21:40:37.135 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.135+0000 7ffa5ca08700 1 -- 172.21.15.196:0/2792852730 shutdown_connections 2020-05-18T21:40:37.136 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.135+0000 7ffa5ca08700 1 --2- 172.21.15.196:0/2792852730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa580a66a0 0x7ffa5809b5f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.136 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.135+0000 7ffa5ca08700 1 -- 172.21.15.196:0/2792852730 >> 172.21.15.196:0/2792852730 conn(0x7ffa58060640 msgr2=0x7ffa580a0260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:37.136 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.135+0000 7ffa5ca08700 1 -- 172.21.15.196:0/2792852730 shutdown_connections 2020-05-18T21:40:37.136 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.135+0000 7ffa5ca08700 1 -- 172.21.15.196:0/2792852730 wait complete. 2020-05-18T21:40:37.146 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2020-05-18T21:40:37.147 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.3 flush_pg_stats 2020-05-18T21:40:37.252 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.251+0000 7f03d880a700 1 Processor -- start 2020-05-18T21:40:37.253 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.251+0000 7f03d880a700 1 -- start start 2020-05-18T21:40:37.256 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d880a700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f03d405b210 0x7f03d40a5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.256 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d880a700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d40a4da0 con 0x7f03d40a6d80 2020-05-18T21:40:37.256 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d880a700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d40668a0 con 0x7f03d40a78f0 2020-05-18T21:40:37.257 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d880a700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d40654e0 con 0x7f03d405b210 2020-05-18T21:40:37.257 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d3fff700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f03d40a78f0 0x7f03d4142d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43302/0 (socket says 172.21.15.196:43302) 2020-05-18T21:40:37.257 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d3fff700 1 -- 172.21.15.196:0/1433747241 learned_addr learned my addr 172.21.15.196:0/1433747241 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:37.257 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d37fe700 1 --2- 172.21.15.196:0/1433747241 >> v2:172.21.15.80:3300/0 conn(0x7f03d405b210 0x7f03d40a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.258 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909904334 0 0) 0x7f03d40668a0 con 0x7f03d40a78f0 2020-05-18T21:40:37.258 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03b4003270 con 0x7f03d40a78f0 2020-05-18T21:40:37.258 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385949257 0 0) 0x7f03d40a4da0 con 0x7f03d40a6d80 2020-05-18T21:40:37.258 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d40668a0 con 0x7f03d40a6d80 2020-05-18T21:40:37.258 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3959648463 0 0) 0x7f03b4003270 con 0x7f03d40a78f0 2020-05-18T21:40:37.259 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 >> v2:172.21.15.80:3300/0 conn(0x7f03d405b210 msgr2=0x7f03d40a5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:37.259 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 --2- 172.21.15.196:0/1433747241 >> v2:172.21.15.80:3300/0 conn(0x7f03d405b210 0x7f03d40a5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.259 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 >> v1:172.21.15.80:6789/0 conn(0x7f03d40a6d80 legacy=0x7f03d40a71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.259 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d4065780 con 0x7f03d40a78f0 2020-05-18T21:40:37.259 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7f03cc001a90 con 0x7f03d40a78f0 2020-05-18T21:40:37.259 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 >> v1:172.21.15.196:6789/0 conn(0x7f03d40a78f0 legacy=0x7f03d4142d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.260 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03b4003940 0x7f03b4003d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.260 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03b4004510 0x7f03b4004990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.260 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4004df0 0x7f03b400b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.260 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40654e0 con 0x7f03b4004510 2020-05-18T21:40:37.260 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b400bbc0 con 0x7f03b4004df0 2020-05-18T21:40:37.261 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b400bd00 con 0x7f03b4003940 2020-05-18T21:40:37.261 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d3fff700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4004df0 0x7f03b400b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.261 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03cbfff700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03b4004510 0x7f03b4004990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.261 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d37fe700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03b4003940 0x7f03b4003d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.261 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d37fe700 1 -- 172.21.15.196:0/1433747241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4004df0 msgr2=0x7f03b400b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:37.262 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d37fe700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4004df0 0x7f03b400b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.262 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d37fe700 1 -- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03b4004510 msgr2=0x7f03b4004990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:37.262 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d37fe700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03b4004510 0x7f03b4004990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.262 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d37fe700 1 -- 172.21.15.196:0/1433747241 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b4003270 con 0x7f03b4003940 2020-05-18T21:40:37.262 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d3fff700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4004df0 0x7f03b400b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:37.262 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d3fff700 1 -- 172.21.15.196:0/1433747241 reap_dead start 2020-05-18T21:40:37.263 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d37fe700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03b4003940 0x7f03b4003d40 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f03c4002f70 tx=0x7f03c4010060).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.263 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d37fe700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03b4003940 0x7f03b4003d40 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f03c4002f70 tx=0x7f03c4010060).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.263 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d37fe700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03b4003940 0x7f03b4003d40 secure :-1 s=READY pgs=24 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).ready entity=mon.2 client_cookie=3413bb7aa6a499a0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.263 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d37fe700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03b4003940 0x7f03b4003d40 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.263 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03c4008d70 con 0x7f03b4003940 2020-05-18T21:40:37.264 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d37fe700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03b4003940 0x7f03b4003d40 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.264 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d37fe700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03b4003940 0x7f03b4003d40 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.264 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c4010910 con 0x7f03b4003940 2020-05-18T21:40:37.264 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d2ffd700 1 -- 172.21.15.196:0/1433747241 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03c4010c70 con 0x7f03b4003940 2020-05-18T21:40:37.264 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d880a700 1 -- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03b4003940 msgr2=0x7f03b4003d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.265 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d880a700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03b4003940 0x7f03b4003d40 secure :-1 s=READY pgs=24 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).stop 2020-05-18T21:40:37.269 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d880a700 1 -- 172.21.15.196:0/1433747241 shutdown_connections 2020-05-18T21:40:37.269 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d880a700 1 --2- 172.21.15.196:0/1433747241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03b4003940 0x7f03b4003d40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.269 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.255+0000 7f03d880a700 1 -- 172.21.15.196:0/1433747241 >> 172.21.15.196:0/1433747241 conn(0x7f03d4060640 msgr2=0x7f03d405a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:37.269 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d880a700 1 -- 172.21.15.196:0/1433747241 shutdown_connections 2020-05-18T21:40:37.270 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d880a700 1 -- 172.21.15.196:0/1433747241 wait complete. 2020-05-18T21:40:37.270 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d880a700 1 Processor -- start 2020-05-18T21:40:37.270 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d880a700 1 -- start start 2020-05-18T21:40:37.270 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d880a700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f03b4003940 0x7f03d40a2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.270 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d880a700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03d40992f0 0x7f03d40996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.271 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d880a700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d409e100 con 0x7f03d409ae00 2020-05-18T21:40:37.271 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d880a700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d405c2d0 con 0x7f03b4003940 2020-05-18T21:40:37.271 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d880a700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d409dea0 con 0x7f03d40992f0 2020-05-18T21:40:37.271 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d37fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f03b4003940 0x7f03d40a2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.271 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d37fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f03b4003940 0x7f03d40a2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34550/0 (socket says 172.21.15.196:34550) 2020-05-18T21:40:37.272 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d37fe700 1 -- 172.21.15.196:0/2633759063 learned_addr learned my addr 172.21.15.196:0/2633759063 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:37.272 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03cbfff700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03d40992f0 0x7f03d40996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.272 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d37fe700 1 -- 172.21.15.196:0/2633759063 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03d40992f0 msgr2=0x7f03d40996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.272 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d37fe700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03d40992f0 0x7f03d40996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.272 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d37fe700 1 -- 172.21.15.196:0/2633759063 >> v1:172.21.15.80:6789/0 conn(0x7f03d409ae00 legacy=0x7f03d409b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.273 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d37fe700 1 -- 172.21.15.196:0/2633759063 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d409d430 con 0x7f03b4003940 2020-05-18T21:40:37.273 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d37fe700 1 --2- 172.21.15.196:0/2633759063 >> v2:172.21.15.196:3300/0 conn(0x7f03b4003940 0x7f03d40a2a80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.273 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d37fe700 1 --2- 172.21.15.196:0/2633759063 >> v2:172.21.15.196:3300/0 conn(0x7f03b4003940 0x7f03d40a2a80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.273 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d37fe700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4003940 0x7f03d40a2a80 secure :-1 s=READY pgs=138 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.273 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d37fe700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4003940 0x7f03d40a2a80 secure :-1 s=THROTTLE_DONE pgs=138 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.274 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d37fe700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4003940 0x7f03d40a2a80 secure :-1 s=THROTTLE_DONE pgs=138 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.274 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d37fe700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4003940 0x7f03d40a2a80 secure :-1 s=THROTTLE_DONE pgs=138 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.274 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.259+0000 7f03d17fa700 1 -- 172.21.15.196:0/2633759063 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03c4005700 con 0x7f03b4003940 2020-05-18T21:40:37.274 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d17fa700 1 -- 172.21.15.196:0/2633759063 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c4005860 con 0x7f03b4003940 2020-05-18T21:40:37.274 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d17fa700 1 -- 172.21.15.196:0/2633759063 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03c4005c00 con 0x7f03b4003940 2020-05-18T21:40:37.275 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d880a700 1 -- 172.21.15.196:0/2633759063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d409cbe0 con 0x7f03b4003940 2020-05-18T21:40:37.275 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d880a700 1 -- 172.21.15.196:0/2633759063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d409c000 con 0x7f03b4003940 2020-05-18T21:40:37.275 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d37fe700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4003940 0x7f03d40a2a80 secure :-1 s=THROTTLE_DONE pgs=138 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.275 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d880a700 1 -- 172.21.15.196:0/2633759063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f03d409db10 con 0x7f03b4003940 2020-05-18T21:40:37.275 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d17fa700 1 -- 172.21.15.196:0/2633759063 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f03c4005d60 con 0x7f03b4003940 2020-05-18T21:40:37.276 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d37fe700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4003940 0x7f03d40a2a80 secure :-1 s=THROTTLE_DONE pgs=138 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.276 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d37fe700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4003940 0x7f03d40a2a80 secure :-1 s=THROTTLE_DONE pgs=138 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.276 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d17fa700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03b804f9e0 0x7f03b8051ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.276 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d17fa700 1 -- 172.21.15.196:0/2633759063 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f03c4063d20 con 0x7f03b4003940 2020-05-18T21:40:37.276 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03cbfff700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03b804f9e0 0x7f03b8051ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.277 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d17fa700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:6824/12775,v1:172.21.15.196:6825/12775] conn(0x7f03b80565c0 0x7f03b80589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.277 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d17fa700 1 -- 172.21.15.196:0/2633759063 --> [v2:172.21.15.196:6824/12775,v1:172.21.15.196:6825/12775] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03b8058fa0 con 0x7f03b80565c0 2020-05-18T21:40:37.277 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d17fa700 1 -- 172.21.15.196:0/2633759063 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03c4065120 con 0x7f03b4003940 2020-05-18T21:40:37.277 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03cbfff700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03b804f9e0 0x7f03b8051ea0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f03bc001110 tx=0x7f03bc003eb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.277 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03cbfff700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03b804f9e0 0x7f03b8051ea0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f03bc001110 tx=0x7f03bc003eb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.278 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03cbfff700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03b804f9e0 0x7f03b8051ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rx=0x7f03bc001110 tx=0x7f03bc003eb0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.278 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d3fff700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:6824/12775,v1:172.21.15.196:6825/12775] conn(0x7f03b80565c0 0x7f03b80589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.278 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.263+0000 7f03d3fff700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:6824/12775,v1:172.21.15.196:6825/12775] conn(0x7f03b80565c0 0x7f03b80589e0 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.278 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.267+0000 7f03d17fa700 1 -- 172.21.15.196:0/2633759063 <== osd.3 v2:172.21.15.196:6824/12775 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23263 (crc 0 0 0) 0x7f03b8058fa0 con 0x7f03b80565c0 2020-05-18T21:40:37.283 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d880a700 1 -- 172.21.15.196:0/2633759063 --> [v2:172.21.15.196:6824/12775,v1:172.21.15.196:6825/12775] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f03d409bb70 con 0x7f03b80565c0 2020-05-18T21:40:37.284 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d17fa700 1 -- 172.21.15.196:0/2633759063 <== osd.3 v2:172.21.15.196:6824/12775 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f03d409bb70 con 0x7f03b80565c0 2020-05-18T21:40:37.285 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d880a700 1 -- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:6824/12775,v1:172.21.15.196:6825/12775] conn(0x7f03b80565c0 msgr2=0x7f03b80589e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.285 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d880a700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:6824/12775,v1:172.21.15.196:6825/12775] conn(0x7f03b80565c0 0x7f03b80589e0 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.285 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d880a700 1 -- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03b804f9e0 msgr2=0x7f03b8051ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.285 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d880a700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f03b804f9e0 0x7f03b8051ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rx=0x7f03bc001110 tx=0x7f03bc003eb0).stop 2020-05-18T21:40:37.286 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d880a700 1 -- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4003940 msgr2=0x7f03d40a2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.286 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d880a700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4003940 0x7f03d40a2a80 secure :-1 s=READY pgs=138 cs=0 l=1 rx=0x7f03c4002f70 tx=0x7f03c4010060).stop 2020-05-18T21:40:37.286 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d3fff700 1 -- 172.21.15.196:0/2633759063 reap_dead start 2020-05-18T21:40:37.286 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d880a700 1 -- 172.21.15.196:0/2633759063 shutdown_connections 2020-05-18T21:40:37.286 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d880a700 1 --2- 172.21.15.196:0/2633759063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03b4003940 0x7f03d40a2a80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.287 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d880a700 1 -- 172.21.15.196:0/2633759063 >> 172.21.15.196:0/2633759063 conn(0x7f03d4060640 msgr2=0x7f03d40a8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:37.287 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d880a700 1 -- 172.21.15.196:0/2633759063 shutdown_connections 2020-05-18T21:40:37.287 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.283+0000 7f03d880a700 1 -- 172.21.15.196:0/2633759063 wait complete. 2020-05-18T21:40:37.295 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2020-05-18T21:40:37.296 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.4 flush_pg_stats 2020-05-18T21:40:37.405 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.403+0000 7f96b5a3f700 1 Processor -- start 2020-05-18T21:40:37.405 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.403+0000 7f96b5a3f700 1 -- start start 2020-05-18T21:40:37.409 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96b5a3f700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b00670c0 0x7f96b0138150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.409 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96b5a3f700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f96b00a6230 0x7f96b00a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.409 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96b5a3f700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b00a8220 con 0x7f96b00a5df0 2020-05-18T21:40:37.410 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96b5a3f700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b00a84c0 con 0x7f96b00a6230 2020-05-18T21:40:37.410 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96b5a3f700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00a8760 con 0x7f96b00670c0 2020-05-18T21:40:37.410 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f96b00a5df0 0x7f96b00a71d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43320/0 (socket says 172.21.15.196:43320) 2020-05-18T21:40:37.410 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 -- 172.21.15.196:0/2288459627 learned_addr learned my addr 172.21.15.196:0/2288459627 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:37.410 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96af7fe700 1 --2- 172.21.15.196:0/2288459627 >> v2:172.21.15.80:3300/0 conn(0x7f96b00a6230 0x7f96b00a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.411 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96aeffd700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b00670c0 0x7f96b0138150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.411 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552312172 0 0) 0x7f96b00a8220 con 0x7f96b00a5df0 2020-05-18T21:40:37.411 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9694003270 con 0x7f96b00a5df0 2020-05-18T21:40:37.411 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 462481483 0 0) 0x7f9694003270 con 0x7f96b00a5df0 2020-05-18T21:40:37.411 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b00670c0 msgr2=0x7f96b0138150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:37.412 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b00670c0 0x7f96b0138150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.412 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 >> v2:172.21.15.80:3300/0 conn(0x7f96b00a6230 msgr2=0x7f96b00a78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:37.412 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 --2- 172.21.15.196:0/2288459627 >> v2:172.21.15.80:3300/0 conn(0x7f96b00a6230 0x7f96b00a78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.412 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00a8a00 con 0x7f96b00a5df0 2020-05-18T21:40:37.412 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7f96a0001710 con 0x7f96b00a5df0 2020-05-18T21:40:37.413 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 >> v1:172.21.15.196:6789/0 conn(0x7f96b00a5df0 legacy=0x7f96b00a71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.413 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9694003760 0x7f9694003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.413 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96940040a0 0x7f9694004570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.413 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b00670c0 0x7f9694004b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.413 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00a84c0 con 0x7f9694003760 2020-05-18T21:40:37.414 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00a8760 con 0x7f96940040a0 2020-05-18T21:40:37.414 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9694005030 con 0x7f96b00670c0 2020-05-18T21:40:37.414 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96940040a0 0x7f9694004570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.414 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96aeffd700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9694003760 0x7f9694003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.414 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96af7fe700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b00670c0 0x7f9694004b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.414 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 -- 172.21.15.196:0/2288459627 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b00670c0 msgr2=0x7f9694004b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:37.415 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b00670c0 0x7f9694004b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.415 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 -- 172.21.15.196:0/2288459627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9694003760 msgr2=0x7f9694003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:37.415 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9694003760 0x7f9694003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.415 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 -- 172.21.15.196:0/2288459627 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9694003270 con 0x7f96940040a0 2020-05-18T21:40:37.415 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96af7fe700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b00670c0 0x7f9694004b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_reply_more state changed! 2020-05-18T21:40:37.416 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96940040a0 0x7f9694004570 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f96a0001420 tx=0x7f96b0139220).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.416 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96940040a0 0x7f9694004570 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f96a0001420 tx=0x7f96b0139220).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.416 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96940040a0 0x7f9694004570 secure :-1 s=READY pgs=140 cs=0 l=1 rx=0x7f96a0001420 tx=0x7f96b0139220).ready entity=mon.1 client_cookie=5e8812f0f508343a server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.416 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96940040a0 0x7f9694004570 secure :-1 s=THROTTLE_DONE pgs=140 cs=0 l=1 rx=0x7f96a0001420 tx=0x7f96b0139220).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.416 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96a000e370 con 0x7f96940040a0 2020-05-18T21:40:37.417 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96940040a0 0x7f9694004570 secure :-1 s=THROTTLE_DONE pgs=140 cs=0 l=1 rx=0x7f96a0001420 tx=0x7f96b0139220).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.417 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96940040a0 0x7f9694004570 secure :-1 s=THROTTLE_DONE pgs=140 cs=0 l=1 rx=0x7f96a0001420 tx=0x7f96b0139220).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.417 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a000e4d0 con 0x7f96940040a0 2020-05-18T21:40:37.417 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96adffb700 1 -- 172.21.15.196:0/2288459627 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96a00100c0 con 0x7f96940040a0 2020-05-18T21:40:37.417 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2288459627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96940040a0 msgr2=0x7f9694004570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.418 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96b5a3f700 1 --2- 172.21.15.196:0/2288459627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96940040a0 0x7f9694004570 secure :-1 s=READY pgs=140 cs=0 l=1 rx=0x7f96a0001420 tx=0x7f96b0139220).stop 2020-05-18T21:40:37.418 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96af7fe700 1 -- 172.21.15.196:0/2288459627 reap_dead start 2020-05-18T21:40:37.418 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2288459627 shutdown_connections 2020-05-18T21:40:37.418 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2288459627 >> 172.21.15.196:0/2288459627 conn(0x7f96b005b290 msgr2=0x7f96b00ac830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:37.418 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2288459627 shutdown_connections 2020-05-18T21:40:37.418 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.407+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2288459627 wait complete. 2020-05-18T21:40:37.422 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.411+0000 7f96b5a3f700 1 Processor -- start 2020-05-18T21:40:37.422 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.411+0000 7f96b5a3f700 1 -- start start 2020-05-18T21:40:37.423 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.411+0000 7f96b5a3f700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f96b013b020 0x7f96b013b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.423 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.411+0000 7f96b5a3f700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b0145800 0x7f96b0145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.423 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.411+0000 7f96b5a3f700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b0135b90 con 0x7f96b00a2c90 2020-05-18T21:40:37.423 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.411+0000 7f96b5a3f700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b013b860 con 0x7f96b013b020 2020-05-18T21:40:37.423 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.411+0000 7f96b5a3f700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b013a230 con 0x7f96b0145800 2020-05-18T21:40:37.424 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.411+0000 7f96af7fe700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f96b00a2c90 0x7f96b00a30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43336/0 (socket says 172.21.15.196:43336) 2020-05-18T21:40:37.424 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.411+0000 7f96af7fe700 1 -- 172.21.15.196:0/2878785044 learned_addr learned my addr 172.21.15.196:0/2878785044 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:37.424 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.411+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b0145800 0x7f96b0145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.424 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.411+0000 7f96aeffd700 1 --2- 172.21.15.196:0/2878785044 >> v2:172.21.15.80:3300/0 conn(0x7f96b013b020 0x7f96b013b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.424 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f969ffff700 1 -- 172.21.15.196:0/2878785044 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2765690442 0 0) 0x7f96b0135b90 con 0x7f96b00a2c90 2020-05-18T21:40:37.425 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f969ffff700 1 -- 172.21.15.196:0/2878785044 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f968c002fe0 con 0x7f96b00a2c90 2020-05-18T21:40:37.425 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96ae7fc700 1 -- 172.21.15.196:0/2878785044 >> v2:172.21.15.80:3300/0 conn(0x7f96b013b020 msgr2=0x7f96b013b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.425 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f969ffff700 1 -- 172.21.15.196:0/2878785044 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 920295582 0 0) 0x7f968c002fe0 con 0x7f96b00a2c90 2020-05-18T21:40:37.425 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2878785044 >> v2:172.21.15.80:3300/0 conn(0x7f96b013b020 0x7f96b013b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.425 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96ae7fc700 1 -- 172.21.15.196:0/2878785044 >> v1:172.21.15.196:6789/0 conn(0x7f96b00a2c90 legacy=0x7f96b00a30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.426 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96ae7fc700 1 -- 172.21.15.196:0/2878785044 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b013a4d0 con 0x7f96b0145800 2020-05-18T21:40:37.426 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b0145800 0x7f96b0145c60 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f96a80019f0 tx=0x7f96b0139220).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.426 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96aeffd700 1 --2- 172.21.15.196:0/2878785044 >> v2:172.21.15.80:3300/0 conn(0x7f96b013b020 0x7f96b013b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:37.426 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b0145800 0x7f96b0145c60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f96a80019f0 tx=0x7f96b0139220).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.426 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b0145800 0x7f96b0145c60 secure :-1 s=READY pgs=25 cs=0 l=1 rx=0x7f96a80019f0 tx=0x7f96b0139220).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b0145800 0x7f96b0145c60 secure :-1 s=THROTTLE_DONE pgs=25 cs=0 l=1 rx=0x7f96a80019f0 tx=0x7f96b0139220).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f969ffff700 1 -- 172.21.15.196:0/2878785044 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96a8001850 con 0x7f96b0145800 2020-05-18T21:40:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b0145800 0x7f96b0145c60 secure :-1 s=THROTTLE_DONE pgs=25 cs=0 l=1 rx=0x7f96a80019f0 tx=0x7f96b0139220).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b0145800 0x7f96b0145c60 secure :-1 s=THROTTLE_DONE pgs=25 cs=0 l=1 rx=0x7f96a80019f0 tx=0x7f96b0139220).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f969ffff700 1 -- 172.21.15.196:0/2878785044 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a000fe80 con 0x7f96b0145800 2020-05-18T21:40:37.427 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2878785044 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b013a770 con 0x7f96b0145800 2020-05-18T21:40:37.428 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f969ffff700 1 -- 172.21.15.196:0/2878785044 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96a000b2c0 con 0x7f96b0145800 2020-05-18T21:40:37.428 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2878785044 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b01460c0 con 0x7f96b0145800 2020-05-18T21:40:37.428 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b0145800 0x7f96b0145c60 secure :-1 s=THROTTLE_DONE pgs=25 cs=0 l=1 rx=0x7f96a80019f0 tx=0x7f96b0139220).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.428 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.415+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2878785044 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f96b00601a0 con 0x7f96b0145800 2020-05-18T21:40:37.428 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f969ffff700 1 -- 172.21.15.196:0/2878785044 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f96a00011d0 con 0x7f96b0145800 2020-05-18T21:40:37.429 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b0145800 0x7f96b0145c60 secure :-1 s=THROTTLE_DONE pgs=25 cs=0 l=1 rx=0x7f96a80019f0 tx=0x7f96b0139220).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.429 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f96ae7fc700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b0145800 0x7f96b0145c60 secure :-1 s=THROTTLE_DONE pgs=25 cs=0 l=1 rx=0x7f96a80019f0 tx=0x7f96b0139220).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.429 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f969ffff700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f968c04fd60 0x7f968c052220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.429 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f969ffff700 1 -- 172.21.15.196:0/2878785044 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f96a0063840 con 0x7f96b0145800 2020-05-18T21:40:37.429 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f96aeffd700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f968c04fd60 0x7f968c052220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.430 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f969ffff700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:6816/13040,v1:172.21.15.80:6819/13040] conn(0x7f968c0568d0 0x7f968c058cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.430 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f969ffff700 1 -- 172.21.15.196:0/2878785044 --> [v2:172.21.15.80:6816/13040,v1:172.21.15.80:6819/13040] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f968c0592b0 con 0x7f968c0568d0 2020-05-18T21:40:37.430 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f969ffff700 1 -- 172.21.15.196:0/2878785044 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96a0063c30 con 0x7f96b0145800 2020-05-18T21:40:37.430 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f96aeffd700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f968c04fd60 0x7f968c052220 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f96a4002f70 tx=0x7f96a400ced0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.430 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f96aeffd700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f968c04fd60 0x7f968c052220 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f96a4002f70 tx=0x7f96a400ced0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f96aeffd700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f968c04fd60 0x7f968c052220 secure :-1 s=READY pgs=53 cs=0 l=1 rx=0x7f96a4002f70 tx=0x7f96a400ced0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f96af7fe700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:6816/13040,v1:172.21.15.80:6819/13040] conn(0x7f968c0568d0 0x7f968c058cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.419+0000 7f96af7fe700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:6816/13040,v1:172.21.15.80:6819/13040] conn(0x7f968c0568d0 0x7f968c058cf0 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.431 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.423+0000 7f969ffff700 1 -- 172.21.15.196:0/2878785044 <== osd.4 v2:172.21.15.80:6816/13040 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23263 (crc 0 0 0) 0x7f968c0592b0 con 0x7f968c0568d0 2020-05-18T21:40:37.439 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.439+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2878785044 --> [v2:172.21.15.80:6816/13040,v1:172.21.15.80:6819/13040] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f96b00559a0 con 0x7f968c0568d0 2020-05-18T21:40:37.440 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.439+0000 7f969ffff700 1 -- 172.21.15.196:0/2878785044 <== osd.4 v2:172.21.15.80:6816/13040 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f96b00559a0 con 0x7f968c0568d0 2020-05-18T21:40:37.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.443+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:6816/13040,v1:172.21.15.80:6819/13040] conn(0x7f968c0568d0 msgr2=0x7f968c058cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.443+0000 7f96b5a3f700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:6816/13040,v1:172.21.15.80:6819/13040] conn(0x7f968c0568d0 0x7f968c058cf0 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.443+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2878785044 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f968c04fd60 msgr2=0x7f968c052220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.443+0000 7f96b5a3f700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f968c04fd60 0x7f968c052220 secure :-1 s=READY pgs=53 cs=0 l=1 rx=0x7f96a4002f70 tx=0x7f96a400ced0).stop 2020-05-18T21:40:37.443 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.443+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b0145800 msgr2=0x7f96b0145c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.443 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.443+0000 7f96b5a3f700 1 --2- 172.21.15.196:0/2878785044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b0145800 0x7f96b0145c60 secure :-1 s=READY pgs=25 cs=0 l=1 rx=0x7f96a80019f0 tx=0x7f96b0139220).stop 2020-05-18T21:40:37.443 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.443+0000 7f96af7fe700 1 -- 172.21.15.196:0/2878785044 reap_dead start 2020-05-18T21:40:37.443 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.443+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2878785044 shutdown_connections 2020-05-18T21:40:37.443 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.443+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2878785044 >> 172.21.15.196:0/2878785044 conn(0x7f96b005b290 msgr2=0x7f96b005eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:37.444 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.443+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2878785044 shutdown_connections 2020-05-18T21:40:37.444 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.443+0000 7f96b5a3f700 1 -- 172.21.15.196:0/2878785044 wait complete. 2020-05-18T21:40:37.453 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2020-05-18T21:40:37.454 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.5 flush_pg_stats 2020-05-18T21:40:37.560 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.559+0000 7fc42f9ef700 1 Processor -- start 2020-05-18T21:40:37.560 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.559+0000 7fc42f9ef700 1 -- start start 2020-05-18T21:40:37.561 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.559+0000 7fc42f9ef700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc4280a1da0 0x7fc428137f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.561 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.559+0000 7fc42f9ef700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc428138370 0x7fc4280666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.562 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.559+0000 7fc42f9ef700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4280a23c0 con 0x7fc428066b40 2020-05-18T21:40:37.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.559+0000 7fc42f9ef700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc428137b20 con 0x7fc428138370 2020-05-18T21:40:37.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.559+0000 7fc42f9ef700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4280a2980 con 0x7fc4280a1da0 2020-05-18T21:40:37.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42cf8a700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc428138370 0x7fc4280666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42df8c700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc428066b40 0x7fc4281345d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43346/0 (socket says 172.21.15.196:43346) 2020-05-18T21:40:37.565 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42df8c700 1 -- 172.21.15.196:0/571960777 learned_addr learned my addr 172.21.15.196:0/571960777 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:37.565 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42d78b700 1 --2- 172.21.15.196:0/571960777 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc4280a1da0 0x7fc428137f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.568 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42cf8a700 1 -- 172.21.15.196:0/571960777 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc4280a1da0 msgr2=0x7fc428137f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:37.569 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42cf8a700 1 --2- 172.21.15.196:0/571960777 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc4280a1da0 0x7fc428137f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.569 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42cf8a700 1 -- 172.21.15.196:0/571960777 >> v1:172.21.15.196:6789/0 conn(0x7fc428066b40 legacy=0x7fc4281345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.570 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc41ffff700 1 -- 172.21.15.196:0/571960777 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606140776 0 0) 0x7fc4280a23c0 con 0x7fc428066b40 2020-05-18T21:40:37.570 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42cf8a700 1 -- 172.21.15.196:0/571960777 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4280600a0 con 0x7fc428138370 2020-05-18T21:40:37.570 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42cf8a700 1 --2- 172.21.15.196:0/571960777 >> v2:172.21.15.196:3300/0 conn(0x7fc428138370 0x7fc4280666e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fc4240032f0 tx=0x7fc424007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.570 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42cf8a700 1 --2- 172.21.15.196:0/571960777 >> v2:172.21.15.196:3300/0 conn(0x7fc428138370 0x7fc4280666e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fc4240032f0 tx=0x7fc424007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.571 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42cf8a700 1 --2- 172.21.15.196:0/571960777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428138370 0x7fc4280666e0 secure :-1 s=READY pgs=143 cs=0 l=1 rx=0x7fc4240032f0 tx=0x7fc424007e00).ready entity=mon.1 client_cookie=67071126e5d39c33 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.571 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42cf8a700 1 --2- 172.21.15.196:0/571960777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428138370 0x7fc4280666e0 secure :-1 s=THROTTLE_DONE pgs=143 cs=0 l=1 rx=0x7fc4240032f0 tx=0x7fc424007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.571 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc41ffff700 1 -- 172.21.15.196:0/571960777 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc424011e60 con 0x7fc428138370 2020-05-18T21:40:37.571 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42cf8a700 1 --2- 172.21.15.196:0/571960777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428138370 0x7fc4280666e0 secure :-1 s=THROTTLE_DONE pgs=143 cs=0 l=1 rx=0x7fc4240032f0 tx=0x7fc424007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.571 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc41ffff700 1 -- 172.21.15.196:0/571960777 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42400a180 con 0x7fc428138370 2020-05-18T21:40:37.572 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42cf8a700 1 --2- 172.21.15.196:0/571960777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428138370 0x7fc4280666e0 secure :-1 s=THROTTLE_DONE pgs=143 cs=0 l=1 rx=0x7fc4240032f0 tx=0x7fc424007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.572 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc41ffff700 1 -- 172.21.15.196:0/571960777 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc42400a520 con 0x7fc428138370 2020-05-18T21:40:37.572 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42f9ef700 1 -- 172.21.15.196:0/571960777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428138370 msgr2=0x7fc4280666e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.572 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42f9ef700 1 --2- 172.21.15.196:0/571960777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428138370 0x7fc4280666e0 secure :-1 s=READY pgs=143 cs=0 l=1 rx=0x7fc4240032f0 tx=0x7fc424007e00).stop 2020-05-18T21:40:37.572 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42f9ef700 1 -- 172.21.15.196:0/571960777 shutdown_connections 2020-05-18T21:40:37.572 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42f9ef700 1 --2- 172.21.15.196:0/571960777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc428138370 0x7fc4280666e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.573 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42f9ef700 1 --2- 172.21.15.196:0/571960777 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc4280a1da0 0x7fc428137f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.573 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42f9ef700 1 -- 172.21.15.196:0/571960777 >> 172.21.15.196:0/571960777 conn(0x7fc42805b290 msgr2=0x7fc4280aa620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:37.573 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42f9ef700 1 -- 172.21.15.196:0/571960777 shutdown_connections 2020-05-18T21:40:37.573 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.563+0000 7fc42f9ef700 1 -- 172.21.15.196:0/571960777 wait complete. 2020-05-18T21:40:37.573 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42f9ef700 1 Processor -- start 2020-05-18T21:40:37.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42f9ef700 1 -- start start 2020-05-18T21:40:37.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42f9ef700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc428138370 0x7fc428138a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42f9ef700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc428141ab0 con 0x7fc428066b40 2020-05-18T21:40:37.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42f9ef700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc428145b00 con 0x7fc4280a1da0 2020-05-18T21:40:37.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42f9ef700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc428148d20 con 0x7fc428138370 2020-05-18T21:40:37.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42cf8a700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc4280a1da0 0x7fc4281453f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43350/0 (socket says 172.21.15.196:43350) 2020-05-18T21:40:37.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42cf8a700 1 -- 172.21.15.196:0/1308960089 learned_addr learned my addr 172.21.15.196:0/1308960089 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:37.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42df8c700 1 --2- 172.21.15.196:0/1308960089 >> v2:172.21.15.80:3300/0 conn(0x7fc428138370 0x7fc428138a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931685262 0 0) 0x7fc428145b00 con 0x7fc4280a1da0 2020-05-18T21:40:37.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc410002fe0 con 0x7fc4280a1da0 2020-05-18T21:40:37.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270214232 0 0) 0x7fc418001f00 con 0x7fc428066b40 2020-05-18T21:40:37.576 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc428145b00 con 0x7fc428066b40 2020-05-18T21:40:37.576 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 2041735264 0 0) 0x7fc410002fe0 con 0x7fc4280a1da0 2020-05-18T21:40:37.576 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42df8c700 1 -- 172.21.15.196:0/1308960089 >> v1:172.21.15.196:6789/0 conn(0x7fc4280a1da0 legacy=0x7fc4281453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.576 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42df8c700 1 -- 172.21.15.196:0/1308960089 >> v1:172.21.15.80:6789/0 conn(0x7fc428066b40 legacy=0x7fc4281413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.576 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42df8c700 1 -- 172.21.15.196:0/1308960089 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc428148fc0 con 0x7fc428138370 2020-05-18T21:40:37.577 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42df8c700 1 --2- 172.21.15.196:0/1308960089 >> v2:172.21.15.80:3300/0 conn(0x7fc428138370 0x7fc428138a90 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc420000bc0 tx=0x7fc4200108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.577 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42df8c700 1 --2- 172.21.15.196:0/1308960089 >> v2:172.21.15.80:3300/0 conn(0x7fc428138370 0x7fc428138a90 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc420000bc0 tx=0x7fc4200108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.577 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42df8c700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc428138370 0x7fc428138a90 secure :-1 s=READY pgs=103 cs=0 l=1 rx=0x7fc420000bc0 tx=0x7fc4200108c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42df8c700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc428138370 0x7fc428138a90 secure :-1 s=THROTTLE_DONE pgs=103 cs=0 l=1 rx=0x7fc420000bc0 tx=0x7fc4200108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc420005ce0 con 0x7fc428138370 2020-05-18T21:40:37.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42df8c700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc428138370 0x7fc428138a90 secure :-1 s=THROTTLE_DONE pgs=103 cs=0 l=1 rx=0x7fc420000bc0 tx=0x7fc4200108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42df8c700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc428138370 0x7fc428138a90 secure :-1 s=THROTTLE_DONE pgs=103 cs=0 l=1 rx=0x7fc420000bc0 tx=0x7fc4200108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc420005e40 con 0x7fc428138370 2020-05-18T21:40:37.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42f9ef700 1 -- 172.21.15.196:0/1308960089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc428149260 con 0x7fc428138370 2020-05-18T21:40:37.579 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4200062c0 con 0x7fc428138370 2020-05-18T21:40:37.579 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.567+0000 7fc42f9ef700 1 -- 172.21.15.196:0/1308960089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4281489a0 con 0x7fc428138370 2020-05-18T21:40:37.581 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc42df8c700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc428138370 0x7fc428138a90 secure :-1 s=THROTTLE_DONE pgs=103 cs=0 l=1 rx=0x7fc420000bc0 tx=0x7fc4200108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.581 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc42f9ef700 1 -- 172.21.15.196:0/1308960089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc4281498a0 con 0x7fc428138370 2020-05-18T21:40:37.581 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fc420006420 con 0x7fc428138370 2020-05-18T21:40:37.581 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc42df8c700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc428138370 0x7fc428138a90 secure :-1 s=THROTTLE_DONE pgs=103 cs=0 l=1 rx=0x7fc420000bc0 tx=0x7fc4200108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.581 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc42df8c700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc428138370 0x7fc428138a90 secure :-1 s=THROTTLE_DONE pgs=103 cs=0 l=1 rx=0x7fc420000bc0 tx=0x7fc4200108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.582 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc41e7fc700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc41004fe30 0x7fc4100522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.582 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc4200090c0 con 0x7fc428138370 2020-05-18T21:40:37.582 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc42d78b700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc41004fe30 0x7fc4100522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.582 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc41e7fc700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:6815/13037,v1:172.21.15.80:6818/13037] conn(0x7fc410056880 0x7fc410058ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.582 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 --> [v2:172.21.15.80:6815/13037,v1:172.21.15.80:6818/13037] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc410059260 con 0x7fc410056880 2020-05-18T21:40:37.583 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc420064d80 con 0x7fc428138370 2020-05-18T21:40:37.583 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc42d78b700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc41004fe30 0x7fc4100522f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc418002f30 tx=0x7fc418002140).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.583 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc42d78b700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc41004fe30 0x7fc4100522f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc418002f30 tx=0x7fc418002140).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.583 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc42d78b700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc41004fe30 0x7fc4100522f0 secure :-1 s=READY pgs=54 cs=0 l=1 rx=0x7fc418002f30 tx=0x7fc418002140).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.583 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.571+0000 7fc42cf8a700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:6815/13037,v1:172.21.15.80:6818/13037] conn(0x7fc410056880 0x7fc410058ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.584 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.575+0000 7fc42cf8a700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:6815/13037,v1:172.21.15.80:6818/13037] conn(0x7fc410056880 0x7fc410058ca0 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.584 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.575+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 <== osd.5 v2:172.21.15.80:6815/13037 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23263 (crc 0 0 0) 0x7fc410059260 con 0x7fc410056880 2020-05-18T21:40:37.593 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc42f9ef700 1 -- 172.21.15.196:0/1308960089 --> [v2:172.21.15.80:6815/13037,v1:172.21.15.80:6818/13037] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc42805c3a0 con 0x7fc410056880 2020-05-18T21:40:37.594 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc41e7fc700 1 -- 172.21.15.196:0/1308960089 <== osd.5 v2:172.21.15.80:6815/13037 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc42805c3a0 con 0x7fc410056880 2020-05-18T21:40:37.595 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc42f9ef700 1 -- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:6815/13037,v1:172.21.15.80:6818/13037] conn(0x7fc410056880 msgr2=0x7fc410058ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.596 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc42f9ef700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:6815/13037,v1:172.21.15.80:6818/13037] conn(0x7fc410056880 0x7fc410058ca0 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.596 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc42f9ef700 1 -- 172.21.15.196:0/1308960089 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc41004fe30 msgr2=0x7fc4100522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.596 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc42f9ef700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc41004fe30 0x7fc4100522f0 secure :-1 s=READY pgs=54 cs=0 l=1 rx=0x7fc418002f30 tx=0x7fc418002140).stop 2020-05-18T21:40:37.596 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc42f9ef700 1 -- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc428138370 msgr2=0x7fc428138a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.597 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc42f9ef700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc428138370 0x7fc428138a90 secure :-1 s=READY pgs=103 cs=0 l=1 rx=0x7fc420000bc0 tx=0x7fc4200108c0).stop 2020-05-18T21:40:37.597 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc42df8c700 1 -- 172.21.15.196:0/1308960089 reap_dead start 2020-05-18T21:40:37.597 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc42f9ef700 1 -- 172.21.15.196:0/1308960089 shutdown_connections 2020-05-18T21:40:37.597 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc42f9ef700 1 --2- 172.21.15.196:0/1308960089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc428138370 0x7fc428138a90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.597 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc42f9ef700 1 -- 172.21.15.196:0/1308960089 >> 172.21.15.196:0/1308960089 conn(0x7fc42805b290 msgr2=0x7fc42805eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:37.597 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc42f9ef700 1 -- 172.21.15.196:0/1308960089 shutdown_connections 2020-05-18T21:40:37.598 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.595+0000 7fc42f9ef700 1 -- 172.21.15.196:0/1308960089 wait complete. 2020-05-18T21:40:37.606 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2020-05-18T21:40:37.607 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.6 flush_pg_stats 2020-05-18T21:40:37.716 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.715+0000 7f894d1ca700 1 Processor -- start 2020-05-18T21:40:37.716 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.715+0000 7f894d1ca700 1 -- start start 2020-05-18T21:40:37.719 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894d1ca700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8948066680 0x7f89480a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.719 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894d1ca700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f894813b1d0 0x7f894805bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.719 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894d1ca700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894805b670 con 0x7f894805baa0 2020-05-18T21:40:37.720 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894d1ca700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89480a9760 con 0x7f894813b1d0 2020-05-18T21:40:37.720 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894d1ca700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89480a9a00 con 0x7f8948066680 2020-05-18T21:40:37.720 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8947fff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8948066680 0x7f89480a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.720 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f89477fe700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f894805baa0 0x7f894813aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43358/0 (socket says 172.21.15.196:43358) 2020-05-18T21:40:37.720 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f89477fe700 1 -- 172.21.15.196:0/160750230 learned_addr learned my addr 172.21.15.196:0/160750230 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894c9c9700 1 --2- 172.21.15.196:0/160750230 >> v2:172.21.15.80:3300/0 conn(0x7f894813b1d0 0x7f894805bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8946ffd700 1 -- 172.21.15.196:0/160750230 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568133647 0 0) 0x7f894805b670 con 0x7f894805baa0 2020-05-18T21:40:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8946ffd700 1 -- 172.21.15.196:0/160750230 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8934003270 con 0x7f894805baa0 2020-05-18T21:40:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8947fff700 1 -- 172.21.15.196:0/160750230 >> v2:172.21.15.80:3300/0 conn(0x7f894813b1d0 msgr2=0x7f894805bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8947fff700 1 --2- 172.21.15.196:0/160750230 >> v2:172.21.15.80:3300/0 conn(0x7f894813b1d0 0x7f894805bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8947fff700 1 -- 172.21.15.196:0/160750230 >> v1:172.21.15.196:6789/0 conn(0x7f894805baa0 legacy=0x7f894813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8946ffd700 1 -- 172.21.15.196:0/160750230 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2127677414 0 0) 0x7f8934003270 con 0x7f894805baa0 2020-05-18T21:40:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8947fff700 1 -- 172.21.15.196:0/160750230 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894805c390 con 0x7f8948066680 2020-05-18T21:40:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894c9c9700 1 --2- 172.21.15.196:0/160750230 >> v2:172.21.15.80:3300/0 conn(0x7f894813b1d0 0x7f894805bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8947fff700 1 --2- 172.21.15.196:0/160750230 >> v2:172.21.15.196:3300/0 conn(0x7f8948066680 0x7f89480a91e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f8938001a30 tx=0x7f894805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8947fff700 1 --2- 172.21.15.196:0/160750230 >> v2:172.21.15.196:3300/0 conn(0x7f8948066680 0x7f89480a91e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f8938001a30 tx=0x7f894805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8947fff700 1 --2- 172.21.15.196:0/160750230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8948066680 0x7f89480a91e0 secure :-1 s=READY pgs=146 cs=0 l=1 rx=0x7f8938001a30 tx=0x7f894805e380).ready entity=mon.1 client_cookie=960964782d2319c7 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8947fff700 1 --2- 172.21.15.196:0/160750230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8948066680 0x7f89480a91e0 secure :-1 s=THROTTLE_DONE pgs=146 cs=0 l=1 rx=0x7f8938001a30 tx=0x7f894805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8946ffd700 1 -- 172.21.15.196:0/160750230 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8938001890 con 0x7f8948066680 2020-05-18T21:40:37.724 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8947fff700 1 --2- 172.21.15.196:0/160750230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8948066680 0x7f89480a91e0 secure :-1 s=THROTTLE_DONE pgs=146 cs=0 l=1 rx=0x7f8938001a30 tx=0x7f894805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.724 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8946ffd700 1 -- 172.21.15.196:0/160750230 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893c008b80 con 0x7f8948066680 2020-05-18T21:40:37.724 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8947fff700 1 --2- 172.21.15.196:0/160750230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8948066680 0x7f89480a91e0 secure :-1 s=THROTTLE_DONE pgs=146 cs=0 l=1 rx=0x7f8938001a30 tx=0x7f894805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.724 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f8946ffd700 1 -- 172.21.15.196:0/160750230 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f893c0113d0 con 0x7f8948066680 2020-05-18T21:40:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894d1ca700 1 -- 172.21.15.196:0/160750230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8948066680 msgr2=0x7f89480a91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.725 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894d1ca700 1 --2- 172.21.15.196:0/160750230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8948066680 0x7f89480a91e0 secure :-1 s=READY pgs=146 cs=0 l=1 rx=0x7f8938001a30 tx=0x7f894805e380).stop 2020-05-18T21:40:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894d1ca700 1 -- 172.21.15.196:0/160750230 shutdown_connections 2020-05-18T21:40:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894d1ca700 1 --2- 172.21.15.196:0/160750230 >> v2:172.21.15.80:3300/0 conn(0x7f894813b1d0 0x7f894805bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.729 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894d1ca700 1 --2- 172.21.15.196:0/160750230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8948066680 0x7f89480a91e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.730 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894d1ca700 1 -- 172.21.15.196:0/160750230 >> 172.21.15.196:0/160750230 conn(0x7f8948069ac0 msgr2=0x7f8948065c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:37.730 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894d1ca700 1 -- 172.21.15.196:0/160750230 shutdown_connections 2020-05-18T21:40:37.730 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.719+0000 7f894d1ca700 1 -- 172.21.15.196:0/160750230 wait complete. 2020-05-18T21:40:37.730 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894d1ca700 1 Processor -- start 2020-05-18T21:40:37.730 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894d1ca700 1 -- start start 2020-05-18T21:40:37.731 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894d1ca700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f894805baa0 0x7f8948138de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.731 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894d1ca700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f894813b1d0 0x7f89480aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.731 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894d1ca700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89480af370 con 0x7f8948066680 2020-05-18T21:40:37.731 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894d1ca700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89480af610 con 0x7f894805baa0 2020-05-18T21:40:37.731 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894d1ca700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89480acd00 con 0x7f894813b1d0 2020-05-18T21:40:37.732 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894c9c9700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f894813b1d0 0x7f89480aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.732 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894c9c9700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f894813b1d0 0x7f89480aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34608/0 (socket says 172.21.15.196:34608) 2020-05-18T21:40:37.732 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894c9c9700 1 -- 172.21.15.196:0/3910277925 learned_addr learned my addr 172.21.15.196:0/3910277925 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:37.732 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f8947fff700 1 --2- 172.21.15.196:0/3910277925 >> v2:172.21.15.80:3300/0 conn(0x7f894805baa0 0x7f8948138de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.732 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894c9c9700 1 -- 172.21.15.196:0/3910277925 >> v2:172.21.15.80:3300/0 conn(0x7f894805baa0 msgr2=0x7f8948138de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.733 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894c9c9700 1 --2- 172.21.15.196:0/3910277925 >> v2:172.21.15.80:3300/0 conn(0x7f894805baa0 0x7f8948138de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.733 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894c9c9700 1 -- 172.21.15.196:0/3910277925 >> v1:172.21.15.80:6789/0 conn(0x7f8948066680 legacy=0x7f89480ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.733 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f89457fa700 1 -- 172.21.15.196:0/3910277925 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417695608 0 0) 0x7f89480af370 con 0x7f8948066680 2020-05-18T21:40:37.733 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894c9c9700 1 -- 172.21.15.196:0/3910277925 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89480ad8b0 con 0x7f894813b1d0 2020-05-18T21:40:37.733 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f8947fff700 1 --2- 172.21.15.196:0/3910277925 >> v2:172.21.15.80:3300/0 conn(0x7f894805baa0 0x7f8948138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:37.734 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894c9c9700 1 --2- 172.21.15.196:0/3910277925 >> v2:172.21.15.196:3300/0 conn(0x7f894813b1d0 0x7f89480aedf0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8940003430 tx=0x7f8940011140).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.734 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.723+0000 7f894c9c9700 1 --2- 172.21.15.196:0/3910277925 >> v2:172.21.15.196:3300/0 conn(0x7f894813b1d0 0x7f89480aedf0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8940003430 tx=0x7f8940011140).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.734 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f894c9c9700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f894813b1d0 0x7f89480aedf0 secure :-1 s=READY pgs=147 cs=0 l=1 rx=0x7f8940003430 tx=0x7f8940011140).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.734 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f894c9c9700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f894813b1d0 0x7f89480aedf0 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f8940003430 tx=0x7f8940011140).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.734 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f89457fa700 1 -- 172.21.15.196:0/3910277925 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8940005e70 con 0x7f894813b1d0 2020-05-18T21:40:37.735 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f894c9c9700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f894813b1d0 0x7f89480aedf0 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f8940003430 tx=0x7f8940011140).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.735 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f894d1ca700 1 -- 172.21.15.196:0/3910277925 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89480a5560 con 0x7f894813b1d0 2020-05-18T21:40:37.735 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f89457fa700 1 -- 172.21.15.196:0/3910277925 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894000cc30 con 0x7f894813b1d0 2020-05-18T21:40:37.735 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f894c9c9700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f894813b1d0 0x7f89480aedf0 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f8940003430 tx=0x7f8940011140).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.735 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f89457fa700 1 -- 172.21.15.196:0/3910277925 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f894000cda0 con 0x7f894813b1d0 2020-05-18T21:40:37.736 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f894d1ca700 1 -- 172.21.15.196:0/3910277925 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89480aca90 con 0x7f894813b1d0 2020-05-18T21:40:37.736 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f894c9c9700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f894813b1d0 0x7f89480aedf0 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f8940003430 tx=0x7f8940011140).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.736 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f894d1ca700 1 -- 172.21.15.196:0/3910277925 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89480acf00 con 0x7f894813b1d0 2020-05-18T21:40:37.736 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f89457fa700 1 -- 172.21.15.196:0/3910277925 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f8940012990 con 0x7f894813b1d0 2020-05-18T21:40:37.736 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f894c9c9700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f894813b1d0 0x7f89480aedf0 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f8940003430 tx=0x7f8940011140).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.737 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f894c9c9700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f894813b1d0 0x7f89480aedf0 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f8940003430 tx=0x7f8940011140).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.737 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f89457fa700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f892804f810 0x7f8928051cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.737 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f89457fa700 1 -- 172.21.15.196:0/3910277925 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8940063d60 con 0x7f894813b1d0 2020-05-18T21:40:37.737 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f8947fff700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f892804f810 0x7f8928051cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.738 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f89457fa700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.80:6802/13039,v1:172.21.15.80:6804/13039] conn(0x7f89280563f0 0x7f8928058810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.738 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f89457fa700 1 -- 172.21.15.196:0/3910277925 --> [v2:172.21.15.80:6802/13039,v1:172.21.15.80:6804/13039] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8928058dd0 con 0x7f89280563f0 2020-05-18T21:40:37.738 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f89457fa700 1 -- 172.21.15.196:0/3910277925 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89400651e0 con 0x7f894813b1d0 2020-05-18T21:40:37.738 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f8947fff700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f892804f810 0x7f8928051cd0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8938001a30 tx=0x7f894805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.738 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f8947fff700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f892804f810 0x7f8928051cd0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8938001a30 tx=0x7f894805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.739 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f8947fff700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f892804f810 0x7f8928051cd0 secure :-1 s=READY pgs=55 cs=0 l=1 rx=0x7f8938001a30 tx=0x7f894805e380).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.739 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.727+0000 7f89477fe700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.80:6802/13039,v1:172.21.15.80:6804/13039] conn(0x7f89280563f0 0x7f8928058810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.739 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.731+0000 7f89477fe700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.80:6802/13039,v1:172.21.15.80:6804/13039] conn(0x7f89280563f0 0x7f8928058810 crc :-1 s=READY pgs=6 cs=0 l=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.740 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.731+0000 7f89457fa700 1 -- 172.21.15.196:0/3910277925 <== osd.6 v2:172.21.15.80:6802/13039 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23263 (crc 0 0 0) 0x7f8928058dd0 con 0x7f89280563f0 2020-05-18T21:40:37.752 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.751+0000 7f894d1ca700 1 -- 172.21.15.196:0/3910277925 --> [v2:172.21.15.80:6802/13039,v1:172.21.15.80:6804/13039] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f89480569e0 con 0x7f89280563f0 2020-05-18T21:40:37.753 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.751+0000 7f89457fa700 1 -- 172.21.15.196:0/3910277925 <== osd.6 v2:172.21.15.80:6802/13039 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f89480569e0 con 0x7f89280563f0 2020-05-18T21:40:37.754 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.755+0000 7f894d1ca700 1 -- 172.21.15.196:0/3910277925 >> [v2:172.21.15.80:6802/13039,v1:172.21.15.80:6804/13039] conn(0x7f89280563f0 msgr2=0x7f8928058810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.754 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.755+0000 7f894d1ca700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.80:6802/13039,v1:172.21.15.80:6804/13039] conn(0x7f89280563f0 0x7f8928058810 crc :-1 s=READY pgs=6 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.755 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.755+0000 7f894d1ca700 1 -- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f892804f810 msgr2=0x7f8928051cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.755 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.755+0000 7f894d1ca700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f892804f810 0x7f8928051cd0 secure :-1 s=READY pgs=55 cs=0 l=1 rx=0x7f8938001a30 tx=0x7f894805e380).stop 2020-05-18T21:40:37.755 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.755+0000 7f894d1ca700 1 -- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f894813b1d0 msgr2=0x7f89480aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.755 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.755+0000 7f894d1ca700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f894813b1d0 0x7f89480aedf0 secure :-1 s=READY pgs=147 cs=0 l=1 rx=0x7f8940003430 tx=0x7f8940011140).stop 2020-05-18T21:40:37.755 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.755+0000 7f894c9c9700 1 -- 172.21.15.196:0/3910277925 reap_dead start 2020-05-18T21:40:37.756 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.755+0000 7f894d1ca700 1 -- 172.21.15.196:0/3910277925 shutdown_connections 2020-05-18T21:40:37.756 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.755+0000 7f894d1ca700 1 --2- 172.21.15.196:0/3910277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f894813b1d0 0x7f89480aedf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.756 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.755+0000 7f894d1ca700 1 -- 172.21.15.196:0/3910277925 >> 172.21.15.196:0/3910277925 conn(0x7f8948069ac0 msgr2=0x7f89480a8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:37.756 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.755+0000 7f894d1ca700 1 -- 172.21.15.196:0/3910277925 shutdown_connections 2020-05-18T21:40:37.756 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.755+0000 7f894d1ca700 1 -- 172.21.15.196:0/3910277925 wait complete. 2020-05-18T21:40:37.770 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2020-05-18T21:40:37.771 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.7 flush_pg_stats 2020-05-18T21:40:37.883 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35cbfa1700 1 Processor -- start 2020-05-18T21:40:37.883 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35cbfa1700 1 -- start start 2020-05-18T21:40:37.886 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35cbfa1700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35c413d230 0x7f35c40a4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.886 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35cbfa1700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c40a6b80 0x7f35c40667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:37.886 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35cbfa1700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c4060140 con 0x7f35c40a2900 2020-05-18T21:40:37.887 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35cbfa1700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c413c730 con 0x7f35c413d230 2020-05-18T21:40:37.887 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35cbfa1700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c40a4580 con 0x7f35c40a6b80 2020-05-18T21:40:37.887 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35c953c700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35c413d230 0x7f35c40a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.887 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35ca53e700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c40a6b80 0x7f35c40667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.887 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35c953c700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35c413d230 0x7f35c40a4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.196:38840/0 (socket says 172.21.15.196:38840) 2020-05-18T21:40:37.888 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35c953c700 1 -- 172.21.15.196:0/1753114841 learned_addr learned my addr 172.21.15.196:0/1753114841 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:37.888 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35ca53e700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c40a6b80 0x7f35c40667b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.196:38450/0 (socket says 172.21.15.196:38450) 2020-05-18T21:40:37.888 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35ca53e700 1 -- 172.21.15.196:0/1753114841 >> v2:172.21.15.80:3300/0 conn(0x7f35c413d230 msgr2=0x7f35c40a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:37.888 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35ca53e700 1 --2- 172.21.15.196:0/1753114841 >> v2:172.21.15.80:3300/0 conn(0x7f35c413d230 0x7f35c40a4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.888 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35ca53e700 1 -- 172.21.15.196:0/1753114841 >> v1:172.21.15.80:6789/0 conn(0x7f35c40a2900 legacy=0x7f35c413cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.889 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35ca53e700 1 -- 172.21.15.196:0/1753114841 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c40a4820 con 0x7f35c40a6b80 2020-05-18T21:40:37.889 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.883+0000 7f35c953c700 1 --2- 172.21.15.196:0/1753114841 >> v2:172.21.15.80:3300/0 conn(0x7f35c413d230 0x7f35c40a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:37.889 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35ca53e700 1 --2- 172.21.15.196:0/1753114841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c40a6b80 0x7f35c40667b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f35bc0032f0 tx=0x7f35bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.889 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35ca53e700 1 --2- 172.21.15.196:0/1753114841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c40a6b80 0x7f35c40667b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f35bc0032f0 tx=0x7f35bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.889 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35ca53e700 1 --2- 172.21.15.196:0/1753114841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c40a6b80 0x7f35c40667b0 secure :-1 s=READY pgs=26 cs=0 l=1 rx=0x7f35bc0032f0 tx=0x7f35bc007e00).ready entity=mon.2 client_cookie=840c2b7465c33e85 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.890 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35ca53e700 1 --2- 172.21.15.196:0/1753114841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c40a6b80 0x7f35c40667b0 secure :-1 s=THROTTLE_DONE pgs=26 cs=0 l=1 rx=0x7f35bc0032f0 tx=0x7f35bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.890 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35c8d3b700 1 -- 172.21.15.196:0/1753114841 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35bc011e60 con 0x7f35c40a6b80 2020-05-18T21:40:37.890 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35ca53e700 1 --2- 172.21.15.196:0/1753114841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c40a6b80 0x7f35c40667b0 secure :-1 s=THROTTLE_DONE pgs=26 cs=0 l=1 rx=0x7f35bc0032f0 tx=0x7f35bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.890 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35c8d3b700 1 -- 172.21.15.196:0/1753114841 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc00a180 con 0x7f35c40a6b80 2020-05-18T21:40:37.890 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35ca53e700 1 --2- 172.21.15.196:0/1753114841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c40a6b80 0x7f35c40667b0 secure :-1 s=THROTTLE_DONE pgs=26 cs=0 l=1 rx=0x7f35bc0032f0 tx=0x7f35bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.891 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35c8d3b700 1 -- 172.21.15.196:0/1753114841 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35bc00a560 con 0x7f35c40a6b80 2020-05-18T21:40:37.891 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35cbfa1700 1 -- 172.21.15.196:0/1753114841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c40a6b80 msgr2=0x7f35c40667b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.891 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35cbfa1700 1 --2- 172.21.15.196:0/1753114841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c40a6b80 0x7f35c40667b0 secure :-1 s=READY pgs=26 cs=0 l=1 rx=0x7f35bc0032f0 tx=0x7f35bc007e00).stop 2020-05-18T21:40:37.894 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35cbfa1700 1 -- 172.21.15.196:0/1753114841 shutdown_connections 2020-05-18T21:40:37.894 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35cbfa1700 1 --2- 172.21.15.196:0/1753114841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c40a6b80 0x7f35c40667b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.895 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35cbfa1700 1 --2- 172.21.15.196:0/1753114841 >> v2:172.21.15.80:3300/0 conn(0x7f35c413d230 0x7f35c40a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:37.895 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35cbfa1700 1 -- 172.21.15.196:0/1753114841 >> 172.21.15.196:0/1753114841 conn(0x7f35c405b290 msgr2=0x7f35c40aa620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:37.895 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35cbfa1700 1 -- 172.21.15.196:0/1753114841 shutdown_connections 2020-05-18T21:40:37.895 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35cbfa1700 1 -- 172.21.15.196:0/1753114841 wait complete. 2020-05-18T21:40:37.895 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35cbfa1700 1 Processor -- start 2020-05-18T21:40:37.896 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.887+0000 7f35cbfa1700 1 -- start start 2020-05-18T21:40:37.896 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35cbfa1700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f35c40a6b80 0x7f35c409dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.896 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35cbfa1700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c413d230 0x7f35c4139b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.896 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35cbfa1700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c409e160 con 0x7f35c40a2900 2020-05-18T21:40:37.896 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35cbfa1700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c409e400 con 0x7f35c40a6b80 2020-05-18T21:40:37.897 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35cbfa1700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c409e6a0 con 0x7f35c413d230 2020-05-18T21:40:37.897 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35c953c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f35c40a6b80 0x7f35c409dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.897 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35c953c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f35c40a6b80 0x7f35c409dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34622/0 (socket says 172.21.15.196:34622) 2020-05-18T21:40:37.897 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35c953c700 1 -- 172.21.15.196:0/2173003242 learned_addr learned my addr 172.21.15.196:0/2173003242 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35ca53e700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c413d230 0x7f35c4139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35c953c700 1 -- 172.21.15.196:0/2173003242 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c413d230 msgr2=0x7f35c4139b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35c953c700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35c413d230 0x7f35c4139b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.898 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35c953c700 1 -- 172.21.15.196:0/2173003242 >> v1:172.21.15.80:6789/0 conn(0x7f35c40a2900 legacy=0x7f35c4143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.899 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35b2ffd700 1 -- 172.21.15.196:0/2173003242 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170377890 0 0) 0x7f35c409e160 con 0x7f35c40a2900 2020-05-18T21:40:37.899 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35c953c700 1 -- 172.21.15.196:0/2173003242 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c4139fd0 con 0x7f35c40a6b80 2020-05-18T21:40:37.899 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35c953c700 1 --2- 172.21.15.196:0/2173003242 >> v2:172.21.15.196:3300/0 conn(0x7f35c40a6b80 0x7f35c409dbe0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f35c0002f70 tx=0x7f35c0010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.899 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35c953c700 1 --2- 172.21.15.196:0/2173003242 >> v2:172.21.15.196:3300/0 conn(0x7f35c40a6b80 0x7f35c409dbe0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f35c0002f70 tx=0x7f35c0010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.899 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35c953c700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40a6b80 0x7f35c409dbe0 secure :-1 s=READY pgs=148 cs=0 l=1 rx=0x7f35c0002f70 tx=0x7f35c0010e80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.900 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35c953c700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40a6b80 0x7f35c409dbe0 secure :-1 s=THROTTLE_DONE pgs=148 cs=0 l=1 rx=0x7f35c0002f70 tx=0x7f35c0010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.900 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35c953c700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40a6b80 0x7f35c409dbe0 secure :-1 s=THROTTLE_DONE pgs=148 cs=0 l=1 rx=0x7f35c0002f70 tx=0x7f35c0010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.900 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35b2ffd700 1 -- 172.21.15.196:0/2173003242 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35c000e370 con 0x7f35c40a6b80 2020-05-18T21:40:37.900 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35c953c700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40a6b80 0x7f35c409dbe0 secure :-1 s=THROTTLE_DONE pgs=148 cs=0 l=1 rx=0x7f35c0002f70 tx=0x7f35c0010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.900 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35b2ffd700 1 -- 172.21.15.196:0/2173003242 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c0012a60 con 0x7f35c40a6b80 2020-05-18T21:40:37.901 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35cbfa1700 1 -- 172.21.15.196:0/2173003242 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c409fcc0 con 0x7f35c40a6b80 2020-05-18T21:40:37.901 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35b2ffd700 1 -- 172.21.15.196:0/2173003242 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35c0012dc0 con 0x7f35c40a6b80 2020-05-18T21:40:37.901 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35cbfa1700 1 -- 172.21.15.196:0/2173003242 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c40a0170 con 0x7f35c40a6b80 2020-05-18T21:40:37.901 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.891+0000 7f35cbfa1700 1 -- 172.21.15.196:0/2173003242 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35c40569e0 con 0x7f35c40a6b80 2020-05-18T21:40:37.901 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35c953c700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40a6b80 0x7f35c409dbe0 secure :-1 s=THROTTLE_DONE pgs=148 cs=0 l=1 rx=0x7f35c0002f70 tx=0x7f35c0010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.902 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35b2ffd700 1 -- 172.21.15.196:0/2173003242 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f35c0010180 con 0x7f35c40a6b80 2020-05-18T21:40:37.902 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35c953c700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40a6b80 0x7f35c409dbe0 secure :-1 s=THROTTLE_DONE pgs=148 cs=0 l=1 rx=0x7f35c0002f70 tx=0x7f35c0010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.902 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35c953c700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40a6b80 0x7f35c409dbe0 secure :-1 s=THROTTLE_DONE pgs=148 cs=0 l=1 rx=0x7f35c0002f70 tx=0x7f35c0010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.903 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35b2ffd700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f35a404f890 0x7f35a4051d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.904 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35c9d3d700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f35a404f890 0x7f35a4051d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.904 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35b2ffd700 1 -- 172.21.15.196:0/2173003242 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f35c0063c90 con 0x7f35c40a6b80 2020-05-18T21:40:37.904 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35b2ffd700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.80:6800/13038,v1:172.21.15.80:6801/13038] conn(0x7f35a4056470 0x7f35a4058890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:37.904 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35b2ffd700 1 -- 172.21.15.196:0/2173003242 --> [v2:172.21.15.80:6800/13038,v1:172.21.15.80:6801/13038] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35a4058e50 con 0x7f35a4056470 2020-05-18T21:40:37.905 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35b2ffd700 1 -- 172.21.15.196:0/2173003242 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35c0063ff0 con 0x7f35c40a6b80 2020-05-18T21:40:37.905 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35c9d3d700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f35a404f890 0x7f35a4051d50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f35b40021b0 tx=0x7f35b4002760).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.905 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35c9d3d700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f35a404f890 0x7f35a4051d50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f35b40021b0 tx=0x7f35b4002760).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:37.905 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35c9d3d700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f35a404f890 0x7f35a4051d50 secure :-1 s=READY pgs=56 cs=0 l=1 rx=0x7f35b40021b0 tx=0x7f35b4002760).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.905 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35ca53e700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.80:6800/13038,v1:172.21.15.80:6801/13038] conn(0x7f35a4056470 0x7f35a4058890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:37.906 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.895+0000 7f35ca53e700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.80:6800/13038,v1:172.21.15.80:6801/13038] conn(0x7f35a4056470 0x7f35a4058890 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:37.906 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.899+0000 7f35b2ffd700 1 -- 172.21.15.196:0/2173003242 <== osd.7 v2:172.21.15.80:6800/13038 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23263 (crc 0 0 0) 0x7f35a4058e50 con 0x7f35a4056470 2020-05-18T21:40:37.918 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35cbfa1700 1 -- 172.21.15.196:0/2173003242 --> [v2:172.21.15.80:6800/13038,v1:172.21.15.80:6801/13038] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f35c40a5fa0 con 0x7f35a4056470 2020-05-18T21:40:37.919 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35b2ffd700 1 -- 172.21.15.196:0/2173003242 <== osd.7 v2:172.21.15.80:6800/13038 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f35c40a5fa0 con 0x7f35a4056470 2020-05-18T21:40:37.920 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35cbfa1700 1 -- 172.21.15.196:0/2173003242 >> [v2:172.21.15.80:6800/13038,v1:172.21.15.80:6801/13038] conn(0x7f35a4056470 msgr2=0x7f35a4058890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.920 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35cbfa1700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.80:6800/13038,v1:172.21.15.80:6801/13038] conn(0x7f35a4056470 0x7f35a4058890 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.921 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35cbfa1700 1 -- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f35a404f890 msgr2=0x7f35a4051d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.921 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35cbfa1700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f35a404f890 0x7f35a4051d50 secure :-1 s=READY pgs=56 cs=0 l=1 rx=0x7f35b40021b0 tx=0x7f35b4002760).stop 2020-05-18T21:40:37.921 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35cbfa1700 1 -- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40a6b80 msgr2=0x7f35c409dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:37.921 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35cbfa1700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40a6b80 0x7f35c409dbe0 secure :-1 s=READY pgs=148 cs=0 l=1 rx=0x7f35c0002f70 tx=0x7f35c0010e80).stop 2020-05-18T21:40:37.921 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35ca53e700 1 -- 172.21.15.196:0/2173003242 reap_dead start 2020-05-18T21:40:37.922 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35cbfa1700 1 -- 172.21.15.196:0/2173003242 shutdown_connections 2020-05-18T21:40:37.922 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35cbfa1700 1 --2- 172.21.15.196:0/2173003242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35c40a6b80 0x7f35c409dbe0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:37.922 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35cbfa1700 1 -- 172.21.15.196:0/2173003242 >> 172.21.15.196:0/2173003242 conn(0x7f35c405b290 msgr2=0x7f35c405eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:37.922 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35cbfa1700 1 -- 172.21.15.196:0/2173003242 shutdown_connections 2020-05-18T21:40:37.922 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:37.919+0000 7f35cbfa1700 1 -- 172.21.15.196:0/2173003242 wait complete. 2020-05-18T21:40:37.932 INFO:teuthology.orchestra.run.smithi196.stdout:55834574853 2020-05-18T21:40:37.933 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-05-18T21:40:38.042 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0eccc8f700 1 Processor -- start 2020-05-18T21:40:38.042 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0eccc8f700 1 -- start start 2020-05-18T21:40:38.045 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0eccc8f700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ec80a7220 0x7f0ec80a3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:38.045 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0eccc8f700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0ec8140840 0x7f0ec8146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:38.045 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0eccc8f700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec809e0b0 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.046 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0eccc8f700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec80a3580 con 0x7f0ec80a7220 2020-05-18T21:40:38.046 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0eccc8f700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec80a20f0 con 0x7f0ec8140840 2020-05-18T21:40:38.046 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec7fff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0ec8140840 0x7f0ec8146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.046 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec7fff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0ec8140840 0x7f0ec8146fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34634/0 (socket says 172.21.15.196:34634) 2020-05-18T21:40:38.047 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec7fff700 1 -- 172.21.15.196:0/1806445500 learned_addr learned my addr 172.21.15.196:0/1806445500 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:38.047 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec77fe700 1 --2- 172.21.15.196:0/1806445500 >> v2:172.21.15.80:3300/0 conn(0x7f0ec80a7220 0x7f0ec80a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.047 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec7fff700 1 -- 172.21.15.196:0/1806445500 >> v2:172.21.15.80:3300/0 conn(0x7f0ec80a7220 msgr2=0x7f0ec80a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:38.047 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec7fff700 1 --2- 172.21.15.196:0/1806445500 >> v2:172.21.15.80:3300/0 conn(0x7f0ec80a7220 0x7f0ec80a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:38.047 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec7fff700 1 -- 172.21.15.196:0/1806445500 >> v1:172.21.15.80:6789/0 conn(0x7f0ec80a3dd0 legacy=0x7f0ec8140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.048 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec7fff700 1 -- 172.21.15.196:0/1806445500 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec80a6a60 con 0x7f0ec8140840 2020-05-18T21:40:38.048 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec77fe700 1 --2- 172.21.15.196:0/1806445500 >> v2:172.21.15.80:3300/0 conn(0x7f0ec80a7220 0x7f0ec80a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_reply_more state changed! 2020-05-18T21:40:38.048 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec7fff700 1 --2- 172.21.15.196:0/1806445500 >> v2:172.21.15.196:3300/0 conn(0x7f0ec8140840 0x7f0ec8146fb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f0ec0002f70 tx=0x7f0ec0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.048 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec7fff700 1 --2- 172.21.15.196:0/1806445500 >> v2:172.21.15.196:3300/0 conn(0x7f0ec8140840 0x7f0ec8146fb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f0ec0002f70 tx=0x7f0ec0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.048 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec7fff700 1 --2- 172.21.15.196:0/1806445500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ec8140840 0x7f0ec8146fb0 secure :-1 s=READY pgs=149 cs=0 l=1 rx=0x7f0ec0002f70 tx=0x7f0ec0007a80).ready entity=mon.1 client_cookie=7c2561d303e04fe server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:38.049 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec7fff700 1 --2- 172.21.15.196:0/1806445500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ec8140840 0x7f0ec8146fb0 secure :-1 s=THROTTLE_DONE pgs=149 cs=0 l=1 rx=0x7f0ec0002f70 tx=0x7f0ec0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.049 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec7fff700 1 --2- 172.21.15.196:0/1806445500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ec8140840 0x7f0ec8146fb0 secure :-1 s=THROTTLE_DONE pgs=149 cs=0 l=1 rx=0x7f0ec0002f70 tx=0x7f0ec0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.049 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec7fff700 1 --2- 172.21.15.196:0/1806445500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ec8140840 0x7f0ec8146fb0 secure :-1 s=THROTTLE_DONE pgs=149 cs=0 l=1 rx=0x7f0ec0002f70 tx=0x7f0ec0007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.049 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec67fc700 1 -- 172.21.15.196:0/1806445500 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec0011760 con 0x7f0ec8140840 2020-05-18T21:40:38.049 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec67fc700 1 -- 172.21.15.196:0/1806445500 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec00118c0 con 0x7f0ec8140840 2020-05-18T21:40:38.050 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1806445500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ec8140840 msgr2=0x7f0ec8146fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.050 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0eccc8f700 1 --2- 172.21.15.196:0/1806445500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ec8140840 0x7f0ec8146fb0 secure :-1 s=READY pgs=149 cs=0 l=1 rx=0x7f0ec0002f70 tx=0x7f0ec0007a80).stop 2020-05-18T21:40:38.050 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0ec67fc700 1 -- 172.21.15.196:0/1806445500 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ec0011ce0 con 0x7f0ec8140840 2020-05-18T21:40:38.050 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1806445500 shutdown_connections 2020-05-18T21:40:38.050 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0eccc8f700 1 --2- 172.21.15.196:0/1806445500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ec8140840 0x7f0ec8146fb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.051 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0eccc8f700 1 --2- 172.21.15.196:0/1806445500 >> v2:172.21.15.80:3300/0 conn(0x7f0ec80a7220 0x7f0ec80a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:38.051 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.043+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1806445500 >> 172.21.15.196:0/1806445500 conn(0x7f0ec805a7c0 msgr2=0x7f0ec805abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:38.051 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1806445500 shutdown_connections 2020-05-18T21:40:38.051 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1806445500 wait complete. 2020-05-18T21:40:38.054 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0eccc8f700 1 Processor -- start 2020-05-18T21:40:38.054 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0eccc8f700 1 -- start start 2020-05-18T21:40:38.055 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0eccc8f700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:38.055 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0eccc8f700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec809d6c0 con 0x7f0ec8140840 2020-05-18T21:40:38.055 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0eccc8f700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec809d8f0 con 0x7f0ec80a7220 2020-05-18T21:40:38.055 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0eccc8f700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec809db90 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.055 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec6ffd700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f0ec80a7220 0x7f0ec813d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43398/0 (socket says 172.21.15.196:43398) 2020-05-18T21:40:38.056 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec6ffd700 1 -- 172.21.15.196:0/1168354057 learned_addr learned my addr 172.21.15.196:0/1168354057 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:38.056 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec77fe700 1 --2- 172.21.15.196:0/1168354057 >> v2:172.21.15.80:3300/0 conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.056 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec4ff9700 1 -- 172.21.15.196:0/1168354057 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323526168 0 0) 0x7f0ec809d8f0 con 0x7f0ec80a7220 2020-05-18T21:40:38.056 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec4ff9700 1 -- 172.21.15.196:0/1168354057 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eac002fe0 con 0x7f0ec80a7220 2020-05-18T21:40:38.057 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec77fe700 1 -- 172.21.15.196:0/1168354057 >> v1:172.21.15.196:6789/0 conn(0x7f0ec80a7220 legacy=0x7f0ec813d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.057 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec77fe700 1 -- 172.21.15.196:0/1168354057 >> v1:172.21.15.80:6789/0 conn(0x7f0ec8140840 legacy=0x7f0ec8149a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.057 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec4ff9700 1 -- 172.21.15.196:0/1168354057 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233110153 0 0) 0x7f0ec809d6c0 con 0x7f0ec8140840 2020-05-18T21:40:38.057 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec77fe700 1 -- 172.21.15.196:0/1168354057 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec809de30 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.057 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec77fe700 1 --2- 172.21.15.196:0/1168354057 >> v2:172.21.15.80:3300/0 conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0eb0002f70 tx=0x7f0eb0010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.058 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec77fe700 1 --2- 172.21.15.196:0/1168354057 >> v2:172.21.15.80:3300/0 conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0eb0002f70 tx=0x7f0eb0010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.058 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec77fe700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 secure :-1 s=READY pgs=109 cs=0 l=1 rx=0x7f0eb0002f70 tx=0x7f0eb0010a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:38.058 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec77fe700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 secure :-1 s=THROTTLE_DONE pgs=109 cs=0 l=1 rx=0x7f0eb0002f70 tx=0x7f0eb0010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.058 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec4ff9700 1 -- 172.21.15.196:0/1168354057 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eb0007180 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.058 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec77fe700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 secure :-1 s=THROTTLE_DONE pgs=109 cs=0 l=1 rx=0x7f0eb0002f70 tx=0x7f0eb0010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.059 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1168354057 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec813db30 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.059 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec4ff9700 1 -- 172.21.15.196:0/1168354057 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb00072e0 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.059 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.047+0000 7f0ec77fe700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 secure :-1 s=THROTTLE_DONE pgs=109 cs=0 l=1 rx=0x7f0eb0002f70 tx=0x7f0eb0010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.059 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.051+0000 7f0ec4ff9700 1 -- 172.21.15.196:0/1168354057 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0eb0012160 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.059 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.051+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1168354057 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec814a2f0 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.060 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.051+0000 7f0ec77fe700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 secure :-1 s=THROTTLE_DONE pgs=109 cs=0 l=1 rx=0x7f0eb0002f70 tx=0x7f0eb0010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.060 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.051+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1168354057 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec81497a0 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.060 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.051+0000 7f0ec4ff9700 1 -- 172.21.15.196:0/1168354057 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f0eb0012360 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.060 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.051+0000 7f0ec77fe700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 secure :-1 s=THROTTLE_DONE pgs=109 cs=0 l=1 rx=0x7f0eb0002f70 tx=0x7f0eb0010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.060 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.051+0000 7f0ec4ff9700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f0eac04fc70 0x7f0eac052130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:38.061 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.051+0000 7f0ec4ff9700 1 -- 172.21.15.196:0/1168354057 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0eb0063b00 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.061 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.051+0000 7f0ec6ffd700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f0eac04fc70 0x7f0eac052130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.061 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.051+0000 7f0ec6ffd700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f0eac04fc70 0x7f0eac052130 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0eb8001520 tx=0x7f0eb8001c00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.061 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.051+0000 7f0ec6ffd700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f0eac04fc70 0x7f0eac052130 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0eb8001520 tx=0x7f0eb8001c00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.061 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.051+0000 7f0ec6ffd700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f0eac04fc70 0x7f0eac052130 secure :-1 s=READY pgs=57 cs=0 l=1 rx=0x7f0eb8001520 tx=0x7f0eb8001c00).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:38.062 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.059+0000 7f0ec77fe700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 secure :-1 s=THROTTLE_DONE pgs=109 cs=0 l=1 rx=0x7f0eb0002f70 tx=0x7f0eb0010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.062 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.059+0000 7f0ec4ff9700 1 -- 172.21.15.196:0/1168354057 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f0ec81497a0 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.183 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.183+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1168354057 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0ec81497a0 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.184 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.183+0000 7f0ec77fe700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 secure :-1 s=THROTTLE_DONE pgs=109 cs=0 l=1 rx=0x7f0eb0002f70 tx=0x7f0eb0010a90).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.184 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.183+0000 7f0ec4ff9700 1 -- 172.21.15.196:0/1168354057 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0ec81497a0 con 0x7f0ec80a3dd0 2020-05-18T21:40:38.185 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2020-05-18T21:40:38.186 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.187+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1168354057 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f0eac04fc70 msgr2=0x7f0eac052130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.186 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.187+0000 7f0eccc8f700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f0eac04fc70 0x7f0eac052130 secure :-1 s=READY pgs=57 cs=0 l=1 rx=0x7f0eb8001520 tx=0x7f0eb8001c00).stop 2020-05-18T21:40:38.187 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.187+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1168354057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ec80a3dd0 msgr2=0x7f0ec80a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.187 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.187+0000 7f0eccc8f700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 secure :-1 s=READY pgs=109 cs=0 l=1 rx=0x7f0eb0002f70 tx=0x7f0eb0010a90).stop 2020-05-18T21:40:38.187 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.187+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1168354057 shutdown_connections 2020-05-18T21:40:38.187 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.187+0000 7f0eccc8f700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f0eac04fc70 0x7f0eac052130 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.187 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.187+0000 7f0eccc8f700 1 --2- 172.21.15.196:0/1168354057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ec80a3dd0 0x7f0ec80a05f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.188 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.187+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1168354057 >> 172.21.15.196:0/1168354057 conn(0x7f0ec805a7c0 msgr2=0x7f0ec805eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:38.188 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.187+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1168354057 shutdown_connections 2020-05-18T21:40:38.188 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.187+0000 7f0eccc8f700 1 -- 172.21.15.196:0/1168354057 wait complete. 2020-05-18T21:40:38.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2020-05-18T21:40:38.201 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.1 2020-05-18T21:40:38.303 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.303+0000 7fca84b09700 1 Processor -- start 2020-05-18T21:40:38.304 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.303+0000 7fca84b09700 1 -- start start 2020-05-18T21:40:38.306 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.303+0000 7fca84b09700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fca800a63f0 0x7fca800a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:38.306 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.303+0000 7fca84b09700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a2b50 0x7fca80132230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:38.306 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.303+0000 7fca84b09700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca800a40b0 con 0x7fca80132690 2020-05-18T21:40:38.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.303+0000 7fca84b09700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca800a1cf0 con 0x7fca800a63f0 2020-05-18T21:40:38.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.303+0000 7fca84b09700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca800a1f90 con 0x7fca800a2b50 2020-05-18T21:40:38.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.303+0000 7fca7ed9d700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fca80132690 0x7fca8005a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43408/0 (socket says 172.21.15.196:43408) 2020-05-18T21:40:38.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.303+0000 7fca7ed9d700 1 -- 172.21.15.196:0/1484959359 learned_addr learned my addr 172.21.15.196:0/1484959359 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:38.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.303+0000 7fca77fff700 1 --2- 172.21.15.196:0/1484959359 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a2b50 0x7fca80132230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.308 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.303+0000 7fca7e59c700 1 --2- 172.21.15.196:0/1484959359 >> v2:172.21.15.80:3300/0 conn(0x7fca800a63f0 0x7fca800a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.308 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.303+0000 7fca7dd9b700 1 -- 172.21.15.196:0/1484959359 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433854356 0 0) 0x7fca800a40b0 con 0x7fca80132690 2020-05-18T21:40:38.308 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.303+0000 7fca7dd9b700 1 -- 172.21.15.196:0/1484959359 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca60003270 con 0x7fca80132690 2020-05-18T21:40:38.308 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca77fff700 1 -- 172.21.15.196:0/1484959359 >> v2:172.21.15.80:3300/0 conn(0x7fca800a63f0 msgr2=0x7fca800a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:38.308 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca77fff700 1 --2- 172.21.15.196:0/1484959359 >> v2:172.21.15.80:3300/0 conn(0x7fca800a63f0 0x7fca800a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:38.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca77fff700 1 -- 172.21.15.196:0/1484959359 >> v1:172.21.15.196:6789/0 conn(0x7fca80132690 legacy=0x7fca8005a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca7dd9b700 1 -- 172.21.15.196:0/1484959359 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2869503821 0 0) 0x7fca60003270 con 0x7fca80132690 2020-05-18T21:40:38.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca77fff700 1 -- 172.21.15.196:0/1484959359 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca800a2230 con 0x7fca800a2b50 2020-05-18T21:40:38.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca77fff700 1 --2- 172.21.15.196:0/1484959359 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a2b50 0x7fca80132230 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fca68002f70 tx=0x7fca68007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca77fff700 1 --2- 172.21.15.196:0/1484959359 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a2b50 0x7fca80132230 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fca68002f70 tx=0x7fca68007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.310 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca77fff700 1 --2- 172.21.15.196:0/1484959359 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a2b50 0x7fca80132230 secure :-1 s=READY pgs=27 cs=0 l=1 rx=0x7fca68002f70 tx=0x7fca68007a80).ready entity=mon.2 client_cookie=2104f2759380dba2 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:38.310 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca77fff700 1 --2- 172.21.15.196:0/1484959359 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a2b50 0x7fca80132230 secure :-1 s=THROTTLE_DONE pgs=27 cs=0 l=1 rx=0x7fca68002f70 tx=0x7fca68007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.310 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca7dd9b700 1 -- 172.21.15.196:0/1484959359 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca68008e80 con 0x7fca800a2b50 2020-05-18T21:40:38.310 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca77fff700 1 --2- 172.21.15.196:0/1484959359 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a2b50 0x7fca80132230 secure :-1 s=THROTTLE_DONE pgs=27 cs=0 l=1 rx=0x7fca68002f70 tx=0x7fca68007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.310 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca77fff700 1 --2- 172.21.15.196:0/1484959359 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a2b50 0x7fca80132230 secure :-1 s=THROTTLE_DONE pgs=27 cs=0 l=1 rx=0x7fca68002f70 tx=0x7fca68007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.311 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca7dd9b700 1 -- 172.21.15.196:0/1484959359 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68011540 con 0x7fca800a2b50 2020-05-18T21:40:38.311 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca7dd9b700 1 -- 172.21.15.196:0/1484959359 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca68011920 con 0x7fca800a2b50 2020-05-18T21:40:38.311 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca84b09700 1 -- 172.21.15.196:0/1484959359 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a2b50 msgr2=0x7fca80132230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.311 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca84b09700 1 --2- 172.21.15.196:0/1484959359 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a2b50 0x7fca80132230 secure :-1 s=READY pgs=27 cs=0 l=1 rx=0x7fca68002f70 tx=0x7fca68007a80).stop 2020-05-18T21:40:38.315 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca84b09700 1 -- 172.21.15.196:0/1484959359 shutdown_connections 2020-05-18T21:40:38.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca84b09700 1 --2- 172.21.15.196:0/1484959359 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a2b50 0x7fca80132230 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca84b09700 1 --2- 172.21.15.196:0/1484959359 >> v2:172.21.15.80:3300/0 conn(0x7fca800a63f0 0x7fca800a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:38.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca84b09700 1 -- 172.21.15.196:0/1484959359 >> 172.21.15.196:0/1484959359 conn(0x7fca800aa600 msgr2=0x7fca800aa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:38.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca84b09700 1 -- 172.21.15.196:0/1484959359 shutdown_connections 2020-05-18T21:40:38.317 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.307+0000 7fca84b09700 1 -- 172.21.15.196:0/1484959359 wait complete. 2020-05-18T21:40:38.317 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca84b09700 1 Processor -- start 2020-05-18T21:40:38.317 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca84b09700 1 -- start start 2020-05-18T21:40:38.317 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca84b09700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a63f0 0x7fca80061930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:38.318 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca84b09700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca80134570 con 0x7fca800a2b50 2020-05-18T21:40:38.318 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca84b09700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca80134810 con 0x7fca80132690 2020-05-18T21:40:38.318 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca84b09700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8009e2c0 con 0x7fca800a63f0 2020-05-18T21:40:38.318 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca7ed9d700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fca80132690 0x7fca80138940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43414/0 (socket says 172.21.15.196:43414) 2020-05-18T21:40:38.318 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca7ed9d700 1 -- 172.21.15.196:0/2894103138 learned_addr learned my addr 172.21.15.196:0/2894103138 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:38.319 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca77fff700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a63f0 0x7fca80061930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.319 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708014758 0 0) 0x7fca80134810 con 0x7fca80132690 2020-05-18T21:40:38.319 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca64002fe0 con 0x7fca80132690 2020-05-18T21:40:38.319 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 76181704 0 0) 0x7fca64002fe0 con 0x7fca80132690 2020-05-18T21:40:38.319 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 --> v1:172.21.15.196:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fca80134810 con 0x7fca80132690 2020-05-18T21:40:38.319 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7fca78001d40 con 0x7fca80132690 2020-05-18T21:40:38.320 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a63f0 msgr2=0x7fca80061930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.320 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a63f0 0x7fca80061930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.320 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 >> v1:172.21.15.196:6789/0 conn(0x7fca80132690 legacy=0x7fca80138940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.320 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 >> v1:172.21.15.80:6789/0 conn(0x7fca800a2b50 legacy=0x7fca80133d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.320 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:38.321 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca640039a0 0x7fca64003e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:38.321 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca64004280 0x7fca6400b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:38.321 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca64005540 con 0x7fca640039a0 2020-05-18T21:40:38.321 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca64004700 con 0x7fca64004a80 2020-05-18T21:40:38.321 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6400bbc0 con 0x7fca64004280 2020-05-18T21:40:38.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca77fff700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca800a63f0 0x7fca80061930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:38.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca7e59c700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca7ed9d700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca64004280 0x7fca6400b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.311+0000 7fca77fff700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca640039a0 0x7fca64003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca7e59c700 1 -- 172.21.15.196:0/2894103138 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca64004280 msgr2=0x7fca6400b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca7e59c700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca64004280 0x7fca6400b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca7e59c700 1 -- 172.21.15.196:0/2894103138 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca640039a0 msgr2=0x7fca64003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca7e59c700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca640039a0 0x7fca64003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca7e59c700 1 -- 172.21.15.196:0/2894103138 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca801384a0 con 0x7fca64004a80 2020-05-18T21:40:38.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca77fff700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca640039a0 0x7fca64003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-05-18T21:40:38.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca84b09700 1 -- 172.21.15.196:0/2894103138 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca800a2640 con 0x7fca64004a80 2020-05-18T21:40:38.324 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca7e59c700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fca70007960 tx=0x7fca640056b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.324 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca84b09700 1 -- 172.21.15.196:0/2894103138 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca801391f0 con 0x7fca64004a80 2020-05-18T21:40:38.324 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca7e59c700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fca70007960 tx=0x7fca640056b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.324 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca7e59c700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 secure :-1 s=READY pgs=153 cs=0 l=1 rx=0x7fca70007960 tx=0x7fca640056b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:38.324 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca7e59c700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7fca70007960 tx=0x7fca640056b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.325 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca70007570 con 0x7fca64004a80 2020-05-18T21:40:38.325 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca7e59c700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7fca70007960 tx=0x7fca640056b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.325 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca7e59c700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7fca70007960 tx=0x7fca640056b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.325 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca70005700 con 0x7fca64004a80 2020-05-18T21:40:38.325 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca70005a60 con 0x7fca64004a80 2020-05-18T21:40:38.325 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca84b09700 1 -- 172.21.15.196:0/2894103138 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca58003380 con 0x7fca64004a80 2020-05-18T21:40:38.326 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca7e59c700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7fca70007960 tx=0x7fca640056b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.326 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fca70005bc0 con 0x7fca64004a80 2020-05-18T21:40:38.326 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca7e59c700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7fca70007960 tx=0x7fca640056b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.327 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca777fe700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fca640582a0 0x7fca6405a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:38.327 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fca70064200 con 0x7fca64004a80 2020-05-18T21:40:38.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca77fff700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fca640582a0 0x7fca6405a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca77fff700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fca640582a0 0x7fca6405a760 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fca68000b60 tx=0x7fca68007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca77fff700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fca640582a0 0x7fca6405a760 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fca68000b60 tx=0x7fca68007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.315+0000 7fca77fff700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fca640582a0 0x7fca6405a760 secure :-1 s=READY pgs=58 cs=0 l=1 rx=0x7fca68000b60 tx=0x7fca68007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:38.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.323+0000 7fca7e59c700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7fca70007960 tx=0x7fca640056b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.323+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7fca7003e650 con 0x7fca64004a80 2020-05-18T21:40:38.437 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.439+0000 7fca84b09700 1 -- 172.21.15.196:0/2894103138 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fca58002210 con 0x7fca64004a80 2020-05-18T21:40:38.438 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.439+0000 7fca7e59c700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7fca70007960 tx=0x7fca640056b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.438 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.439+0000 7fca777fe700 1 -- 172.21.15.196:0/2894103138 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fca70041c70 con 0x7fca64004a80 2020-05-18T21:40:38.438 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2020-05-18T21:40:38.440 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.439+0000 7fca84b09700 1 -- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fca640582a0 msgr2=0x7fca6405a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.439+0000 7fca84b09700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fca640582a0 0x7fca6405a760 secure :-1 s=READY pgs=58 cs=0 l=1 rx=0x7fca68000b60 tx=0x7fca68007a80).stop 2020-05-18T21:40:38.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.439+0000 7fca84b09700 1 -- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 msgr2=0x7fca64004e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.439+0000 7fca84b09700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 secure :-1 s=READY pgs=153 cs=0 l=1 rx=0x7fca70007960 tx=0x7fca640056b0).stop 2020-05-18T21:40:38.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.439+0000 7fca7ed9d700 1 -- 172.21.15.196:0/2894103138 reap_dead start 2020-05-18T21:40:38.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.439+0000 7fca84b09700 1 -- 172.21.15.196:0/2894103138 shutdown_connections 2020-05-18T21:40:38.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.439+0000 7fca84b09700 1 --2- 172.21.15.196:0/2894103138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64004a80 0x7fca64004e60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.439+0000 7fca84b09700 1 -- 172.21.15.196:0/2894103138 >> 172.21.15.196:0/2894103138 conn(0x7fca800aa600 msgr2=0x7fca8005eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:38.443 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.443+0000 7fca84b09700 1 -- 172.21.15.196:0/2894103138 shutdown_connections 2020-05-18T21:40:38.443 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.443+0000 7fca84b09700 1 -- 172.21.15.196:0/2894103138 wait complete. 2020-05-18T21:40:38.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2020-05-18T21:40:38.455 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.2 2020-05-18T21:40:38.560 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.559+0000 7f3974e76700 1 Processor -- start 2020-05-18T21:40:38.560 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.559+0000 7f3974e76700 1 -- start start 2020-05-18T21:40:38.562 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.559+0000 7f3974e76700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f39700a1f10 0x7f39700a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:38.562 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.559+0000 7f3974e76700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3970137ce0 0x7f3970138130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:38.562 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.559+0000 7f3974e76700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39700a6d20 con 0x7f39700a7b20 2020-05-18T21:40:38.563 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.559+0000 7f3974e76700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39700a40c0 con 0x7f39700a1f10 2020-05-18T21:40:38.563 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.559+0000 7f3974e76700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39700a4360 con 0x7f3970137ce0 2020-05-18T21:40:38.563 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.559+0000 7f3967fff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f39700a1f10 0x7f39700a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.563 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.559+0000 7f3967fff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f39700a1f10 0x7f39700a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34664/0 (socket says 172.21.15.196:34664) 2020-05-18T21:40:38.563 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.559+0000 7f3967fff700 1 -- 172.21.15.196:0/2121919857 learned_addr learned my addr 172.21.15.196:0/2121919857 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:38.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.559+0000 7f396ffff700 1 --2- 172.21.15.196:0/2121919857 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3970137ce0 0x7f3970138130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3967fff700 1 -- 172.21.15.196:0/2121919857 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3970137ce0 msgr2=0x7f3970138130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:38.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3967fff700 1 --2- 172.21.15.196:0/2121919857 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3970137ce0 0x7f3970138130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:38.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3967fff700 1 -- 172.21.15.196:0/2121919857 >> v1:172.21.15.80:6789/0 conn(0x7f39700a7b20 legacy=0x7f39700a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:38.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3967fff700 1 -- 172.21.15.196:0/2121919857 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39700672b0 con 0x7f39700a1f10 2020-05-18T21:40:38.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f396ffff700 1 --2- 172.21.15.196:0/2121919857 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3970137ce0 0x7f3970138130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request state changed! 2020-05-18T21:40:38.565 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3967fff700 1 --2- 172.21.15.196:0/2121919857 >> v2:172.21.15.196:3300/0 conn(0x7f39700a1f10 0x7f39700a68c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f397005fd50 tx=0x7f3958007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.565 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3967fff700 1 --2- 172.21.15.196:0/2121919857 >> v2:172.21.15.196:3300/0 conn(0x7f39700a1f10 0x7f39700a68c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f397005fd50 tx=0x7f3958007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.565 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3967fff700 1 --2- 172.21.15.196:0/2121919857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a68c0 secure :-1 s=READY pgs=154 cs=0 l=1 rx=0x7f397005fd50 tx=0x7f3958007a80).ready entity=mon.1 client_cookie=15320b04bc1b15d7 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:38.565 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3967fff700 1 --2- 172.21.15.196:0/2121919857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a68c0 secure :-1 s=THROTTLE_DONE pgs=154 cs=0 l=1 rx=0x7f397005fd50 tx=0x7f3958007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.565 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3967fff700 1 --2- 172.21.15.196:0/2121919857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a68c0 secure :-1 s=THROTTLE_DONE pgs=154 cs=0 l=1 rx=0x7f397005fd50 tx=0x7f3958007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.566 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f396effd700 1 -- 172.21.15.196:0/2121919857 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3958011720 con 0x7f39700a1f10 2020-05-18T21:40:38.566 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3967fff700 1 --2- 172.21.15.196:0/2121919857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a68c0 secure :-1 s=THROTTLE_DONE pgs=154 cs=0 l=1 rx=0x7f397005fd50 tx=0x7f3958007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.566 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f396effd700 1 -- 172.21.15.196:0/2121919857 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3958011880 con 0x7f39700a1f10 2020-05-18T21:40:38.566 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f396effd700 1 -- 172.21.15.196:0/2121919857 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3958011ca0 con 0x7f39700a1f10 2020-05-18T21:40:38.566 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 -- 172.21.15.196:0/2121919857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 msgr2=0x7f39700a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.567 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 --2- 172.21.15.196:0/2121919857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a68c0 secure :-1 s=READY pgs=154 cs=0 l=1 rx=0x7f397005fd50 tx=0x7f3958007a80).stop 2020-05-18T21:40:38.569 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 -- 172.21.15.196:0/2121919857 shutdown_connections 2020-05-18T21:40:38.570 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 --2- 172.21.15.196:0/2121919857 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3970137ce0 0x7f3970138130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:38.570 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 --2- 172.21.15.196:0/2121919857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a68c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.570 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 -- 172.21.15.196:0/2121919857 >> 172.21.15.196:0/2121919857 conn(0x7f397005a7c0 msgr2=0x7f397005abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:38.570 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 -- 172.21.15.196:0/2121919857 shutdown_connections 2020-05-18T21:40:38.570 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 -- 172.21.15.196:0/2121919857 wait complete. 2020-05-18T21:40:38.571 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 Processor -- start 2020-05-18T21:40:38.571 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 -- start start 2020-05-18T21:40:38.571 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f39700a1f10 0x7f39700a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:38.571 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39700a7b20 0x7f397014bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:38.571 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39700669a0 con 0x7f3970137ce0 2020-05-18T21:40:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f397012b720 con 0x7f39700a1f10 2020-05-18T21:40:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3974e76700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397012cdd0 con 0x7f39700a7b20 2020-05-18T21:40:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f396f7fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f39700a1f10 0x7f39700a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f396f7fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f39700a1f10 0x7f39700a8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34670/0 (socket says 172.21.15.196:34670) 2020-05-18T21:40:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f396f7fe700 1 -- 172.21.15.196:0/4099257005 learned_addr learned my addr 172.21.15.196:0/4099257005 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.563+0000 7f3967fff700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39700a7b20 0x7f397014bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396f7fe700 1 -- 172.21.15.196:0/4099257005 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39700a7b20 msgr2=0x7f397014bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396f7fe700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39700a7b20 0x7f397014bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396f7fe700 1 -- 172.21.15.196:0/4099257005 >> v1:172.21.15.196:6789/0 conn(0x7f3970137ce0 legacy=0x7f397014e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396f7fe700 1 -- 172.21.15.196:0/4099257005 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39700a47d0 con 0x7f39700a1f10 2020-05-18T21:40:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396f7fe700 1 --2- 172.21.15.196:0/4099257005 >> v2:172.21.15.196:3300/0 conn(0x7f39700a1f10 0x7f39700a8c50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f3960001f90 tx=0x7f3960010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396f7fe700 1 --2- 172.21.15.196:0/4099257005 >> v2:172.21.15.196:3300/0 conn(0x7f39700a1f10 0x7f39700a8c50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f3960001f90 tx=0x7f3960010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396f7fe700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a8c50 secure :-1 s=READY pgs=156 cs=0 l=1 rx=0x7f3960001f90 tx=0x7f3960010a50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:38.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396f7fe700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a8c50 secure :-1 s=THROTTLE_DONE pgs=156 cs=0 l=1 rx=0x7f3960001f90 tx=0x7f3960010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396f7fe700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a8c50 secure :-1 s=THROTTLE_DONE pgs=156 cs=0 l=1 rx=0x7f3960001f90 tx=0x7f3960010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396f7fe700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a8c50 secure :-1 s=THROTTLE_DONE pgs=156 cs=0 l=1 rx=0x7f3960001f90 tx=0x7f3960010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396d7fa700 1 -- 172.21.15.196:0/4099257005 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f396000cd50 con 0x7f39700a1f10 2020-05-18T21:40:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396d7fa700 1 -- 172.21.15.196:0/4099257005 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396000e180 con 0x7f39700a1f10 2020-05-18T21:40:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f3974e76700 1 -- 172.21.15.196:0/4099257005 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397012b280 con 0x7f39700a1f10 2020-05-18T21:40:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396d7fa700 1 -- 172.21.15.196:0/4099257005 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f396000e520 con 0x7f39700a1f10 2020-05-18T21:40:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f3974e76700 1 -- 172.21.15.196:0/4099257005 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397009a860 con 0x7f39700a1f10 2020-05-18T21:40:38.576 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396f7fe700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a8c50 secure :-1 s=THROTTLE_DONE pgs=156 cs=0 l=1 rx=0x7f3960001f90 tx=0x7f3960010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.576 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396f7fe700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a8c50 secure :-1 s=THROTTLE_DONE pgs=156 cs=0 l=1 rx=0x7f3960001f90 tx=0x7f3960010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.576 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396d7fa700 1 -- 172.21.15.196:0/4099257005 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f396000e790 con 0x7f39700a1f10 2020-05-18T21:40:38.576 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f3974e76700 1 -- 172.21.15.196:0/4099257005 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397012c3b0 con 0x7f39700a1f10 2020-05-18T21:40:38.577 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396d7fa700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f395404f980 0x7f3954051e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:38.577 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f396d7fa700 1 -- 172.21.15.196:0/4099257005 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3960063c20 con 0x7f39700a1f10 2020-05-18T21:40:38.577 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f3967fff700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f395404f980 0x7f3954051e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.577 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f3967fff700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f395404f980 0x7f3954051e40 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f397005fd50 tx=0x7f3958007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f3967fff700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f395404f980 0x7f3954051e40 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f397005fd50 tx=0x7f3958007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.567+0000 7f3967fff700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f395404f980 0x7f3954051e40 secure :-1 s=READY pgs=59 cs=0 l=1 rx=0x7f397005fd50 tx=0x7f3958007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:38.578 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.571+0000 7f396f7fe700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a8c50 secure :-1 s=THROTTLE_DONE pgs=156 cs=0 l=1 rx=0x7f3960001f90 tx=0x7f3960010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.579 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.571+0000 7f396d7fa700 1 -- 172.21.15.196:0/4099257005 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f396003df10 con 0x7f39700a1f10 2020-05-18T21:40:38.693 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.695+0000 7f3974e76700 1 -- 172.21.15.196:0/4099257005 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f39700569e0 con 0x7f39700a1f10 2020-05-18T21:40:38.694 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.695+0000 7f396f7fe700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a8c50 secure :-1 s=THROTTLE_DONE pgs=156 cs=0 l=1 rx=0x7f3960001f90 tx=0x7f3960010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.695 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.695+0000 7f396d7fa700 1 -- 172.21.15.196:0/4099257005 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3960041530 con 0x7f39700a1f10 2020-05-18T21:40:38.695 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2020-05-18T21:40:38.697 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.695+0000 7f3974e76700 1 -- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f395404f980 msgr2=0x7f3954051e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.697 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.695+0000 7f3974e76700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f395404f980 0x7f3954051e40 secure :-1 s=READY pgs=59 cs=0 l=1 rx=0x7f397005fd50 tx=0x7f3958007a80).stop 2020-05-18T21:40:38.697 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.695+0000 7f3974e76700 1 -- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 msgr2=0x7f39700a8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.698 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.695+0000 7f3974e76700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a8c50 secure :-1 s=READY pgs=156 cs=0 l=1 rx=0x7f3960001f90 tx=0x7f3960010a50).stop 2020-05-18T21:40:38.698 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.695+0000 7f3974e76700 1 -- 172.21.15.196:0/4099257005 shutdown_connections 2020-05-18T21:40:38.698 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.695+0000 7f3974e76700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f395404f980 0x7f3954051e40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.698 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.695+0000 7f3974e76700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39700a7b20 0x7f397014bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.698 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.695+0000 7f3974e76700 1 --2- 172.21.15.196:0/4099257005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39700a1f10 0x7f39700a8c50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.698 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.695+0000 7f3974e76700 1 -- 172.21.15.196:0/4099257005 >> 172.21.15.196:0/4099257005 conn(0x7f397005a7c0 msgr2=0x7f39700a6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:38.699 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.699+0000 7f3974e76700 1 -- 172.21.15.196:0/4099257005 shutdown_connections 2020-05-18T21:40:38.699 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.699+0000 7f3974e76700 1 -- 172.21.15.196:0/4099257005 wait complete. 2020-05-18T21:40:38.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2020-05-18T21:40:38.712 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.3 2020-05-18T21:40:38.815 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c1aa6a700 1 Processor -- start 2020-05-18T21:40:38.815 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c1aa6a700 1 -- start start 2020-05-18T21:40:38.817 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c1aa6a700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1c140a6800 0x7f1c140a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:38.817 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c1aa6a700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c140a3f10 con 0x7f1c1405ab00 2020-05-18T21:40:38.818 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c1aa6a700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1405b5c0 con 0x7f1c140a5330 2020-05-18T21:40:38.818 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c1aa6a700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c140a5e50 con 0x7f1c140a6800 2020-05-18T21:40:38.818 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c19007700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f1c140a5330 0x7f1c140a5740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43440/0 (socket says 172.21.15.196:43440) 2020-05-18T21:40:38.822 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c13fff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1c140a6800 0x7f1c140a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.823 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c19007700 1 -- 172.21.15.196:0/3932474619 learned_addr learned my addr 172.21.15.196:0/3932474619 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:38.823 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c137fe700 1 -- 172.21.15.196:0/3932474619 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173879190 0 0) 0x7f1c1405b5c0 con 0x7f1c140a5330 2020-05-18T21:40:38.823 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c137fe700 1 -- 172.21.15.196:0/3932474619 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c00003270 con 0x7f1c140a5330 2020-05-18T21:40:38.823 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c13fff700 1 -- 172.21.15.196:0/3932474619 >> v1:172.21.15.196:6789/0 conn(0x7f1c140a5330 legacy=0x7f1c140a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.823 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c13fff700 1 -- 172.21.15.196:0/3932474619 >> v1:172.21.15.80:6789/0 conn(0x7f1c1405ab00 legacy=0x7f1c140a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c13fff700 1 -- 172.21.15.196:0/3932474619 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1405b860 con 0x7f1c140a6800 2020-05-18T21:40:38.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c13fff700 1 --2- 172.21.15.196:0/3932474619 >> v2:172.21.15.196:3300/0 conn(0x7f1c140a6800 0x7f1c140a4ed0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f1c08002f70 tx=0x7f1c08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c13fff700 1 --2- 172.21.15.196:0/3932474619 >> v2:172.21.15.196:3300/0 conn(0x7f1c140a6800 0x7f1c140a4ed0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f1c08002f70 tx=0x7f1c08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c13fff700 1 --2- 172.21.15.196:0/3932474619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c140a6800 0x7f1c140a4ed0 secure :-1 s=READY pgs=158 cs=0 l=1 rx=0x7f1c08002f70 tx=0x7f1c08007a80).ready entity=mon.1 client_cookie=a306370f4dfb4bbf server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:38.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c13fff700 1 --2- 172.21.15.196:0/3932474619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c140a6800 0x7f1c140a4ed0 secure :-1 s=THROTTLE_DONE pgs=158 cs=0 l=1 rx=0x7f1c08002f70 tx=0x7f1c08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c137fe700 1 -- 172.21.15.196:0/3932474619 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c08008e80 con 0x7f1c140a6800 2020-05-18T21:40:38.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.815+0000 7f1c13fff700 1 --2- 172.21.15.196:0/3932474619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c140a6800 0x7f1c140a4ed0 secure :-1 s=THROTTLE_DONE pgs=158 cs=0 l=1 rx=0x7f1c08002f70 tx=0x7f1c08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c137fe700 1 -- 172.21.15.196:0/3932474619 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c080115f0 con 0x7f1c140a6800 2020-05-18T21:40:38.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c13fff700 1 --2- 172.21.15.196:0/3932474619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c140a6800 0x7f1c140a4ed0 secure :-1 s=THROTTLE_DONE pgs=158 cs=0 l=1 rx=0x7f1c08002f70 tx=0x7f1c08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.826 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3932474619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c140a6800 msgr2=0x7f1c140a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.826 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c137fe700 1 -- 172.21.15.196:0/3932474619 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c08011990 con 0x7f1c140a6800 2020-05-18T21:40:38.826 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 --2- 172.21.15.196:0/3932474619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c140a6800 0x7f1c140a4ed0 secure :-1 s=READY pgs=158 cs=0 l=1 rx=0x7f1c08002f70 tx=0x7f1c08007a80).stop 2020-05-18T21:40:38.831 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3932474619 shutdown_connections 2020-05-18T21:40:38.831 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 --2- 172.21.15.196:0/3932474619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c140a6800 0x7f1c140a4ed0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.831 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3932474619 >> 172.21.15.196:0/3932474619 conn(0x7f1c140aa600 msgr2=0x7f1c140aa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:38.831 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3932474619 shutdown_connections 2020-05-18T21:40:38.832 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3932474619 wait complete. 2020-05-18T21:40:38.832 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 Processor -- start 2020-05-18T21:40:38.832 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 -- start start 2020-05-18T21:40:38.832 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1c1405ab00 0x7f1c14099090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:38.832 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1c140a6800 0x7f1c1414cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:38.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1409d7c0 con 0x7f1c140a5330 2020-05-18T21:40:38.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c140a4700 con 0x7f1c140a6800 2020-05-18T21:40:38.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c1aa6a700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1409e870 con 0x7f1c1405ab00 2020-05-18T21:40:38.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c13fff700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f1c140a5330 0x7f1c1414d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43444/0 (socket says 172.21.15.196:43444) 2020-05-18T21:40:38.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c13fff700 1 -- 172.21.15.196:0/3105307898 learned_addr learned my addr 172.21.15.196:0/3105307898 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:38.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c18806700 1 --2- 172.21.15.196:0/3105307898 >> v2:172.21.15.196:3300/0 conn(0x7f1c1405ab00 0x7f1c14099090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.819+0000 7f1c19007700 1 --2- 172.21.15.196:0/3105307898 >> v2:172.21.15.80:3300/0 conn(0x7f1c140a6800 0x7f1c1414cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c11ffb700 1 -- 172.21.15.196:0/3105307898 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390063463 0 0) 0x7f1c1409d7c0 con 0x7f1c140a5330 2020-05-18T21:40:38.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c11ffb700 1 -- 172.21.15.196:0/3105307898 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bf4002fe0 con 0x7f1c140a5330 2020-05-18T21:40:38.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c18806700 1 -- 172.21.15.196:0/3105307898 >> v2:172.21.15.80:3300/0 conn(0x7f1c140a6800 msgr2=0x7f1c1414cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c18806700 1 --2- 172.21.15.196:0/3105307898 >> v2:172.21.15.80:3300/0 conn(0x7f1c140a6800 0x7f1c1414cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c11ffb700 1 -- 172.21.15.196:0/3105307898 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1352693868 0 0) 0x7f1bf4002fe0 con 0x7f1c140a5330 2020-05-18T21:40:38.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c18806700 1 -- 172.21.15.196:0/3105307898 >> v1:172.21.15.196:6789/0 conn(0x7f1c140a5330 legacy=0x7f1c1414d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c18806700 1 -- 172.21.15.196:0/3105307898 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1409a3b0 con 0x7f1c1405ab00 2020-05-18T21:40:38.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c18806700 1 --2- 172.21.15.196:0/3105307898 >> v2:172.21.15.196:3300/0 conn(0x7f1c1405ab00 0x7f1c14099090 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1c08008360 tx=0x7f1c0c010c40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c18806700 1 --2- 172.21.15.196:0/3105307898 >> v2:172.21.15.196:3300/0 conn(0x7f1c1405ab00 0x7f1c14099090 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1c08008360 tx=0x7f1c0c010c40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c18806700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c1405ab00 0x7f1c14099090 secure :-1 s=READY pgs=160 cs=0 l=1 rx=0x7f1c08008360 tx=0x7f1c0c010c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:38.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c18806700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c1405ab00 0x7f1c14099090 secure :-1 s=THROTTLE_DONE pgs=160 cs=0 l=1 rx=0x7f1c08008360 tx=0x7f1c0c010c40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c18806700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c1405ab00 0x7f1c14099090 secure :-1 s=THROTTLE_DONE pgs=160 cs=0 l=1 rx=0x7f1c08008360 tx=0x7f1c0c010c40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c18806700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c1405ab00 0x7f1c14099090 secure :-1 s=THROTTLE_DONE pgs=160 cs=0 l=1 rx=0x7f1c08008360 tx=0x7f1c0c010c40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c11ffb700 1 -- 172.21.15.196:0/3105307898 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c08008e80 con 0x7f1c1405ab00 2020-05-18T21:40:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c11ffb700 1 -- 172.21.15.196:0/3105307898 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c006d90 con 0x7f1c1405ab00 2020-05-18T21:40:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3105307898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c14097970 con 0x7f1c1405ab00 2020-05-18T21:40:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c11ffb700 1 -- 172.21.15.196:0/3105307898 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c0c002450 con 0x7f1c1405ab00 2020-05-18T21:40:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3105307898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c140a13d0 con 0x7f1c1405ab00 2020-05-18T21:40:38.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c18806700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c1405ab00 0x7f1c14099090 secure :-1 s=THROTTLE_DONE pgs=160 cs=0 l=1 rx=0x7f1c08008360 tx=0x7f1c0c010c40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3105307898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c140569e0 con 0x7f1c1405ab00 2020-05-18T21:40:38.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c11ffb700 1 -- 172.21.15.196:0/3105307898 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f1c0c003080 con 0x7f1c1405ab00 2020-05-18T21:40:38.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c18806700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c1405ab00 0x7f1c14099090 secure :-1 s=THROTTLE_DONE pgs=160 cs=0 l=1 rx=0x7f1c08008360 tx=0x7f1c0c010c40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c11ffb700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1bf404fde0 0x7f1bf40522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:38.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c11ffb700 1 -- 172.21.15.196:0/3105307898 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1c0c0636b0 con 0x7f1c1405ab00 2020-05-18T21:40:38.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c13fff700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1bf404fde0 0x7f1bf40522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:38.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c13fff700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1bf404fde0 0x7f1bf40522a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1c08002f10 tx=0x7f1c08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c13fff700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1bf404fde0 0x7f1bf40522a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1c08002f10 tx=0x7f1c08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.823+0000 7f1c13fff700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1bf404fde0 0x7f1bf40522a0 secure :-1 s=READY pgs=60 cs=0 l=1 rx=0x7f1c08002f10 tx=0x7f1c08007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:38.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.831+0000 7f1c18806700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c1405ab00 0x7f1c14099090 secure :-1 s=THROTTLE_DONE pgs=160 cs=0 l=1 rx=0x7f1c08008360 tx=0x7f1c0c010c40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.831+0000 7f1c11ffb700 1 -- 172.21.15.196:0/3105307898 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f1c0c03daa0 con 0x7f1c1405ab00 2020-05-18T21:40:38.945 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.947+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3105307898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1c140a3f10 con 0x7f1c1405ab00 2020-05-18T21:40:38.948 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.947+0000 7f1c18806700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c1405ab00 0x7f1c14099090 secure :-1 s=THROTTLE_DONE pgs=160 cs=0 l=1 rx=0x7f1c08008360 tx=0x7f1c0c010c40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:38.948 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.947+0000 7f1c11ffb700 1 -- 172.21.15.196:0/3105307898 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1c140a3f10 con 0x7f1c1405ab00 2020-05-18T21:40:38.949 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2020-05-18T21:40:38.950 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.951+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1bf404fde0 msgr2=0x7f1bf40522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.951 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.951+0000 7f1c1aa6a700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1bf404fde0 0x7f1bf40522a0 secure :-1 s=READY pgs=60 cs=0 l=1 rx=0x7f1c08002f10 tx=0x7f1c08007a80).stop 2020-05-18T21:40:38.951 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.951+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c1405ab00 msgr2=0x7f1c14099090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:38.951 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.951+0000 7f1c1aa6a700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c1405ab00 0x7f1c14099090 secure :-1 s=READY pgs=160 cs=0 l=1 rx=0x7f1c08008360 tx=0x7f1c0c010c40).stop 2020-05-18T21:40:38.951 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.951+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3105307898 shutdown_connections 2020-05-18T21:40:38.952 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.951+0000 7f1c1aa6a700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1bf404fde0 0x7f1bf40522a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.952 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.951+0000 7f1c1aa6a700 1 --2- 172.21.15.196:0/3105307898 >> v2:172.21.15.80:3300/0 conn(0x7f1c140a6800 0x7f1c1414cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.952 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.951+0000 7f1c1aa6a700 1 --2- 172.21.15.196:0/3105307898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c1405ab00 0x7f1c14099090 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:38.952 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.951+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3105307898 >> 172.21.15.196:0/3105307898 conn(0x7f1c140aa600 msgr2=0x7f1c1405eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:38.952 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.951+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3105307898 shutdown_connections 2020-05-18T21:40:38.952 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:38.951+0000 7f1c1aa6a700 1 -- 172.21.15.196:0/3105307898 wait complete. 2020-05-18T21:40:38.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2020-05-18T21:40:38.961 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.4 2020-05-18T21:40:39.059 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.059+0000 7f62c0867700 1 Processor -- start 2020-05-18T21:40:39.060 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.059+0000 7f62c0867700 1 -- start start 2020-05-18T21:40:39.061 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62c0867700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc0ab910 0x7f62bc0a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:39.061 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62c0867700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62bc137ff0 con 0x7f62bc134cf0 2020-05-18T21:40:39.062 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62c0867700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62bc138250 con 0x7f62bc137bb0 2020-05-18T21:40:39.062 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62c0867700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc0a66d0 con 0x7f62bc0ab910 2020-05-18T21:40:39.062 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62b9d9b700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f62bc137bb0 0x7f62bc1345e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43456/0 (socket says 172.21.15.196:43456) 2020-05-18T21:40:39.062 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62b9d9b700 1 -- 172.21.15.196:0/4208831338 learned_addr learned my addr 172.21.15.196:0/4208831338 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:39.062 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62ba59c700 1 --2- 172.21.15.196:0/4208831338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc0ab910 0x7f62bc0a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.064 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62b959a700 1 -- 172.21.15.196:0/4208831338 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591638456 0 0) 0x7f62bc138250 con 0x7f62bc137bb0 2020-05-18T21:40:39.064 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62b959a700 1 -- 172.21.15.196:0/4208831338 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62a0003270 con 0x7f62bc137bb0 2020-05-18T21:40:39.065 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62b959a700 1 -- 172.21.15.196:0/4208831338 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741255646 0 0) 0x7f62bc137ff0 con 0x7f62bc134cf0 2020-05-18T21:40:39.065 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62b959a700 1 -- 172.21.15.196:0/4208831338 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62bc138250 con 0x7f62bc134cf0 2020-05-18T21:40:39.065 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62ba59c700 1 -- 172.21.15.196:0/4208831338 >> v1:172.21.15.196:6789/0 conn(0x7f62bc137bb0 legacy=0x7f62bc1345e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.065 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62ba59c700 1 -- 172.21.15.196:0/4208831338 >> v1:172.21.15.80:6789/0 conn(0x7f62bc134cf0 legacy=0x7f62bc12e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.065 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62ba59c700 1 -- 172.21.15.196:0/4208831338 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc1384f0 con 0x7f62bc0ab910 2020-05-18T21:40:39.066 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62ba59c700 1 --2- 172.21.15.196:0/4208831338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc0ab910 0x7f62bc0a5fd0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f62ac002f70 tx=0x7f62ac008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.066 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62ba59c700 1 --2- 172.21.15.196:0/4208831338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc0ab910 0x7f62bc0a5fd0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f62ac002f70 tx=0x7f62ac008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.066 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62ba59c700 1 --2- 172.21.15.196:0/4208831338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc0ab910 0x7f62bc0a5fd0 secure :-1 s=READY pgs=28 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62ac008920).ready entity=mon.2 client_cookie=7dd2d652429ff4a8 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:39.066 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62ba59c700 1 --2- 172.21.15.196:0/4208831338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc0ab910 0x7f62bc0a5fd0 secure :-1 s=THROTTLE_DONE pgs=28 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62ac008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.066 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62b959a700 1 -- 172.21.15.196:0/4208831338 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62ac008e80 con 0x7f62bc0ab910 2020-05-18T21:40:39.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62ba59c700 1 --2- 172.21.15.196:0/4208831338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc0ab910 0x7f62bc0a5fd0 secure :-1 s=THROTTLE_DONE pgs=28 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62ac008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62b959a700 1 -- 172.21.15.196:0/4208831338 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac011580 con 0x7f62bc0ab910 2020-05-18T21:40:39.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62ba59c700 1 --2- 172.21.15.196:0/4208831338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc0ab910 0x7f62bc0a5fd0 secure :-1 s=THROTTLE_DONE pgs=28 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62ac008920).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62b959a700 1 -- 172.21.15.196:0/4208831338 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62ac0118e0 con 0x7f62bc0ab910 2020-05-18T21:40:39.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62c0867700 1 -- 172.21.15.196:0/4208831338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc0ab910 msgr2=0x7f62bc0a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62c0867700 1 --2- 172.21.15.196:0/4208831338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc0ab910 0x7f62bc0a5fd0 secure :-1 s=READY pgs=28 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62ac008920).stop 2020-05-18T21:40:39.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62c0867700 1 -- 172.21.15.196:0/4208831338 shutdown_connections 2020-05-18T21:40:39.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62c0867700 1 --2- 172.21.15.196:0/4208831338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc0ab910 0x7f62bc0a5fd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62c0867700 1 -- 172.21.15.196:0/4208831338 >> 172.21.15.196:0/4208831338 conn(0x7f62bc060fe0 msgr2=0x7f62bc0aaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:39.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62c0867700 1 -- 172.21.15.196:0/4208831338 shutdown_connections 2020-05-18T21:40:39.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.063+0000 7f62c0867700 1 -- 172.21.15.196:0/4208831338 wait complete. 2020-05-18T21:40:39.071 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62c0867700 1 Processor -- start 2020-05-18T21:40:39.071 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62c0867700 1 -- start start 2020-05-18T21:40:39.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62c0867700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f62bc0ab910 0x7f62bc0a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:39.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62c0867700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc134cf0 0x7f62bc12f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:39.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62c0867700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62bc1357b0 con 0x7f62bc137bb0 2020-05-18T21:40:39.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62c0867700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62bc144430 con 0x7f62bc0ab910 2020-05-18T21:40:39.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62c0867700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc131280 con 0x7f62bc134cf0 2020-05-18T21:40:39.073 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62b9d9b700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc134cf0 0x7f62bc12f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.073 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ba59c700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f62bc0ab910 0x7f62bc0a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.073 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62b9d9b700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc134cf0 0x7f62bc12f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.196:38534/0 (socket says 172.21.15.196:38534) 2020-05-18T21:40:39.073 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62b9d9b700 1 -- 172.21.15.196:0/3907527374 learned_addr learned my addr 172.21.15.196:0/3907527374 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:39.073 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ba59c700 1 -- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc134cf0 msgr2=0x7f62bc12f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.074 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ba59c700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc134cf0 0x7f62bc12f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.074 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ba59c700 1 -- 172.21.15.196:0/3907527374 >> v1:172.21.15.80:6789/0 conn(0x7f62bc137bb0 legacy=0x7f62bc144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.074 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ba59c700 1 -- 172.21.15.196:0/3907527374 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc131520 con 0x7f62bc0ab910 2020-05-18T21:40:39.074 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62b9d9b700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc134cf0 0x7f62bc12f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:39.074 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ba59c700 1 --2- 172.21.15.196:0/3907527374 >> v2:172.21.15.80:3300/0 conn(0x7f62bc0ab910 0x7f62bc0a4680 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62bc135750).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.075 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ba59c700 1 --2- 172.21.15.196:0/3907527374 >> v2:172.21.15.80:3300/0 conn(0x7f62bc0ab910 0x7f62bc0a4680 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62bc135750).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.075 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ba59c700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62bc0ab910 0x7f62bc0a4680 secure :-1 s=READY pgs=115 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62bc135750).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:39.075 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ba59c700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62bc0ab910 0x7f62bc0a4680 secure :-1 s=THROTTLE_DONE pgs=115 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62bc135750).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.075 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ab7fe700 1 -- 172.21.15.196:0/3907527374 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62ac007170 con 0x7f62bc0ab910 2020-05-18T21:40:39.075 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62c0867700 1 -- 172.21.15.196:0/3907527374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62bc131800 con 0x7f62bc0ab910 2020-05-18T21:40:39.075 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ba59c700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62bc0ab910 0x7f62bc0a4680 secure :-1 s=THROTTLE_DONE pgs=115 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62bc135750).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.076 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ab7fe700 1 -- 172.21.15.196:0/3907527374 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac007310 con 0x7f62bc0ab910 2020-05-18T21:40:39.076 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62c0867700 1 -- 172.21.15.196:0/3907527374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62bc134380 con 0x7f62bc0ab910 2020-05-18T21:40:39.076 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ba59c700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62bc0ab910 0x7f62bc0a4680 secure :-1 s=THROTTLE_DONE pgs=115 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62bc135750).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.076 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ab7fe700 1 -- 172.21.15.196:0/3907527374 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62ac005700 con 0x7f62bc0ab910 2020-05-18T21:40:39.076 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.067+0000 7f62ba59c700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62bc0ab910 0x7f62bc0a4680 secure :-1 s=THROTTLE_DONE pgs=115 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62bc135750).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.077 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.071+0000 7f62c0867700 1 -- 172.21.15.196:0/3907527374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62bc0569e0 con 0x7f62bc0ab910 2020-05-18T21:40:39.077 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.071+0000 7f62ab7fe700 1 -- 172.21.15.196:0/3907527374 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f62ac0059c0 con 0x7f62bc0ab910 2020-05-18T21:40:39.077 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.071+0000 7f62ba59c700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62bc0ab910 0x7f62bc0a4680 secure :-1 s=THROTTLE_DONE pgs=115 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62bc135750).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.077 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.071+0000 7f62ab7fe700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f629804f9c0 0x7f6298051e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:39.077 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.071+0000 7f62ab7fe700 1 -- 172.21.15.196:0/3907527374 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f62ac064710 con 0x7f62bc0ab910 2020-05-18T21:40:39.078 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.071+0000 7f62b9d9b700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f629804f9c0 0x7f6298051e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.078 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.071+0000 7f62b9d9b700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f629804f9c0 0x7f6298051e80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f62b0001110 tx=0x7f62b0001410).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.078 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.071+0000 7f62b9d9b700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f629804f9c0 0x7f6298051e80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f62b0001110 tx=0x7f62b0001410).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.078 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.071+0000 7f62b9d9b700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f629804f9c0 0x7f6298051e80 secure :-1 s=READY pgs=62 cs=0 l=1 rx=0x7f62b0001110 tx=0x7f62b0001410).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:39.079 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.079+0000 7f62ba59c700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62bc0ab910 0x7f62bc0a4680 secure :-1 s=THROTTLE_DONE pgs=115 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62bc135750).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.079 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.079+0000 7f62ab7fe700 1 -- 172.21.15.196:0/3907527374 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f62ac03eaa0 con 0x7f62bc0ab910 2020-05-18T21:40:39.191 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.191+0000 7f62c0867700 1 -- 172.21.15.196:0/3907527374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f62bc068f40 con 0x7f62bc0ab910 2020-05-18T21:40:39.192 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.191+0000 7f62ba59c700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62bc0ab910 0x7f62bc0a4680 secure :-1 s=THROTTLE_DONE pgs=115 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62bc135750).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.192 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.191+0000 7f62ab7fe700 1 -- 172.21.15.196:0/3907527374 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f62ac0420c0 con 0x7f62bc0ab910 2020-05-18T21:40:39.193 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2020-05-18T21:40:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.195+0000 7f62c0867700 1 -- 172.21.15.196:0/3907527374 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f629804f9c0 msgr2=0x7f6298051e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.195+0000 7f62c0867700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f629804f9c0 0x7f6298051e80 secure :-1 s=READY pgs=62 cs=0 l=1 rx=0x7f62b0001110 tx=0x7f62b0001410).stop 2020-05-18T21:40:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.195+0000 7f62c0867700 1 -- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62bc0ab910 msgr2=0x7f62bc0a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.195+0000 7f62c0867700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62bc0ab910 0x7f62bc0a4680 secure :-1 s=READY pgs=115 cs=0 l=1 rx=0x7f62ac002f70 tx=0x7f62bc135750).stop 2020-05-18T21:40:39.195 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.195+0000 7f62c0867700 1 -- 172.21.15.196:0/3907527374 shutdown_connections 2020-05-18T21:40:39.196 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.195+0000 7f62c0867700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f629804f9c0 0x7f6298051e80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.196 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.195+0000 7f62c0867700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62bc134cf0 0x7f62bc12f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.196 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.195+0000 7f62c0867700 1 --2- 172.21.15.196:0/3907527374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62bc0ab910 0x7f62bc0a4680 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.196 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.195+0000 7f62c0867700 1 -- 172.21.15.196:0/3907527374 >> 172.21.15.196:0/3907527374 conn(0x7f62bc060fe0 msgr2=0x7f62bc05b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:39.196 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.195+0000 7f62c0867700 1 -- 172.21.15.196:0/3907527374 shutdown_connections 2020-05-18T21:40:39.197 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.195+0000 7f62c0867700 1 -- 172.21.15.196:0/3907527374 wait complete. 2020-05-18T21:40:39.204 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2020-05-18T21:40:39.205 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.5 2020-05-18T21:40:39.305 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.303+0000 7f3352c6a700 1 Processor -- start 2020-05-18T21:40:39.305 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.303+0000 7f3352c6a700 1 -- start start 2020-05-18T21:40:39.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352c6a700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f334c0a63d0 0x7f334c134b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:39.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352c6a700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f334c0a34e0 0x7f334c134fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:39.307 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352c6a700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c0a6d40 con 0x7f334c0ab740 2020-05-18T21:40:39.308 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352c6a700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c09e700 con 0x7f334c0a63d0 2020-05-18T21:40:39.308 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352c6a700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0a57d0 con 0x7f334c0a34e0 2020-05-18T21:40:39.308 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352469700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f334c0a34e0 0x7f334c134fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.308 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351467700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f334c0a63d0 0x7f334c134b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.308 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351467700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f334c0a63d0 0x7f334c134b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.196:38934/0 (socket says 172.21.15.196:38934) 2020-05-18T21:40:39.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352469700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f334c0a34e0 0x7f334c134fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.196:38544/0 (socket says 172.21.15.196:38544) 2020-05-18T21:40:39.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351467700 1 -- 172.21.15.196:0/2653833070 learned_addr learned my addr 172.21.15.196:0/2653833070 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:39.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351c68700 1 --1- 172.21.15.196:0/2653833070 >> v1:172.21.15.80:6789/0 conn(0x7f334c0ab740 0x7f334c0a2dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.196:39296/0 (socket says 172.21.15.196:39296) 2020-05-18T21:40:39.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351467700 1 -- 172.21.15.196:0/2653833070 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f334c0a34e0 msgr2=0x7f334c134fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:39.309 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351467700 1 --2- 172.21.15.196:0/2653833070 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f334c0a34e0 0x7f334c134fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:39.310 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351467700 1 -- 172.21.15.196:0/2653833070 >> v1:172.21.15.80:6789/0 conn(0x7f334c0ab740 legacy=0x7f334c0a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.310 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351467700 1 -- 172.21.15.196:0/2653833070 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c0a5a70 con 0x7f334c0a63d0 2020-05-18T21:40:39.310 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351467700 1 --2- 172.21.15.196:0/2653833070 >> v2:172.21.15.80:3300/0 conn(0x7f334c0a63d0 0x7f334c134b60 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f333c002f70 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.310 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351467700 1 --2- 172.21.15.196:0/2653833070 >> v2:172.21.15.80:3300/0 conn(0x7f334c0a63d0 0x7f334c134b60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f333c002f70 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.310 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351467700 1 --2- 172.21.15.196:0/2653833070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c0a63d0 0x7f334c134b60 secure :-1 s=READY pgs=117 cs=0 l=1 rx=0x7f333c002f70 tx=0x7f333c007a80).ready entity=mon.0 client_cookie=e193cef057a25ae5 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:39.310 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351467700 1 --2- 172.21.15.196:0/2653833070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c0a63d0 0x7f334c134b60 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7f333c002f70 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.311 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3350c66700 1 -- 172.21.15.196:0/2653833070 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f333c0116b0 con 0x7f334c0a63d0 2020-05-18T21:40:39.311 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351467700 1 --2- 172.21.15.196:0/2653833070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c0a63d0 0x7f334c134b60 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7f333c002f70 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.311 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3350c66700 1 -- 172.21.15.196:0/2653833070 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c011810 con 0x7f334c0a63d0 2020-05-18T21:40:39.311 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3351467700 1 --2- 172.21.15.196:0/2653833070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c0a63d0 0x7f334c134b60 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7f333c002f70 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.311 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352c6a700 1 -- 172.21.15.196:0/2653833070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c0a63d0 msgr2=0x7f334c134b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.312 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3350c66700 1 -- 172.21.15.196:0/2653833070 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f333c011c30 con 0x7f334c0a63d0 2020-05-18T21:40:39.312 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352c6a700 1 --2- 172.21.15.196:0/2653833070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c0a63d0 0x7f334c134b60 secure :-1 s=READY pgs=117 cs=0 l=1 rx=0x7f333c002f70 tx=0x7f333c007a80).stop 2020-05-18T21:40:39.315 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352c6a700 1 -- 172.21.15.196:0/2653833070 shutdown_connections 2020-05-18T21:40:39.315 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352c6a700 1 --2- 172.21.15.196:0/2653833070 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f334c0a34e0 0x7f334c134fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:39.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352c6a700 1 --2- 172.21.15.196:0/2653833070 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f334c0a63d0 0x7f334c134b60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352c6a700 1 -- 172.21.15.196:0/2653833070 >> 172.21.15.196:0/2653833070 conn(0x7f334c060e20 msgr2=0x7f334c0aacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:39.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352c6a700 1 -- 172.21.15.196:0/2653833070 shutdown_connections 2020-05-18T21:40:39.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.307+0000 7f3352c6a700 1 -- 172.21.15.196:0/2653833070 wait complete. 2020-05-18T21:40:39.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3352c6a700 1 Processor -- start 2020-05-18T21:40:39.317 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3352c6a700 1 -- start start 2020-05-18T21:40:39.317 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3352c6a700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f334c0a63d0 0x7f334c129760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:39.317 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3352c6a700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f334c0ab740 0x7f334c148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:39.317 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3352c6a700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c0a16a0 con 0x7f334c0a34e0 2020-05-18T21:40:39.317 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3352c6a700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c0a1160 con 0x7f334c0ab740 2020-05-18T21:40:39.318 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3352c6a700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c0a1400 con 0x7f334c0a63d0 2020-05-18T21:40:39.318 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351467700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f334c0a63d0 0x7f334c129760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.318 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351c68700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f334c0a34e0 0x7f334c126de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43474/0 (socket says 172.21.15.196:43474) 2020-05-18T21:40:39.318 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351c68700 1 -- 172.21.15.196:0/3526958859 learned_addr learned my addr 172.21.15.196:0/3526958859 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:39.318 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3352469700 1 --2- 172.21.15.196:0/3526958859 >> v2:172.21.15.80:3300/0 conn(0x7f334c0ab740 0x7f334c148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.318 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351467700 1 -- 172.21.15.196:0/3526958859 >> v2:172.21.15.80:3300/0 conn(0x7f334c0ab740 msgr2=0x7f334c148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.319 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f333affd700 1 -- 172.21.15.196:0/3526958859 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940659134 0 0) 0x7f334c0a16a0 con 0x7f334c0a34e0 2020-05-18T21:40:39.319 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351467700 1 --2- 172.21.15.196:0/3526958859 >> v2:172.21.15.80:3300/0 conn(0x7f334c0ab740 0x7f334c148a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.319 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351467700 1 -- 172.21.15.196:0/3526958859 >> v1:172.21.15.196:6789/0 conn(0x7f334c0a34e0 legacy=0x7f334c126de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.319 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351467700 1 -- 172.21.15.196:0/3526958859 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c0667c0 con 0x7f334c0a63d0 2020-05-18T21:40:39.319 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351467700 1 --2- 172.21.15.196:0/3526958859 >> v2:172.21.15.196:3300/0 conn(0x7f334c0a63d0 0x7f334c129760 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f334c0a0490 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.320 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351467700 1 --2- 172.21.15.196:0/3526958859 >> v2:172.21.15.196:3300/0 conn(0x7f334c0a63d0 0x7f334c129760 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f334c0a0490 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.320 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351467700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c0a63d0 0x7f334c129760 secure :-1 s=READY pgs=163 cs=0 l=1 rx=0x7f334c0a0490 tx=0x7f333c007a80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:39.320 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351467700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c0a63d0 0x7f334c129760 secure :-1 s=THROTTLE_DONE pgs=163 cs=0 l=1 rx=0x7f334c0a0490 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.320 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f333affd700 1 -- 172.21.15.196:0/3526958859 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f333c011cb0 con 0x7f334c0a63d0 2020-05-18T21:40:39.320 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351467700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c0a63d0 0x7f334c129760 secure :-1 s=THROTTLE_DONE pgs=163 cs=0 l=1 rx=0x7f334c0a0490 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.321 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f333affd700 1 -- 172.21.15.196:0/3526958859 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c007360 con 0x7f334c0a63d0 2020-05-18T21:40:39.321 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3352c6a700 1 -- 172.21.15.196:0/3526958859 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c135800 con 0x7f334c0a63d0 2020-05-18T21:40:39.321 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351467700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c0a63d0 0x7f334c129760 secure :-1 s=THROTTLE_DONE pgs=163 cs=0 l=1 rx=0x7f334c0a0490 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.321 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f333affd700 1 -- 172.21.15.196:0/3526958859 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f333c005700 con 0x7f334c0a63d0 2020-05-18T21:40:39.321 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3352c6a700 1 -- 172.21.15.196:0/3526958859 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c09f2a0 con 0x7f334c0a63d0 2020-05-18T21:40:39.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.311+0000 7f3351467700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c0a63d0 0x7f334c129760 secure :-1 s=THROTTLE_DONE pgs=163 cs=0 l=1 rx=0x7f334c0a0490 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.315+0000 7f3352c6a700 1 -- 172.21.15.196:0/3526958859 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c0569e0 con 0x7f334c0a63d0 2020-05-18T21:40:39.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.315+0000 7f333affd700 1 -- 172.21.15.196:0/3526958859 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f333c0058b0 con 0x7f334c0a63d0 2020-05-18T21:40:39.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.315+0000 7f3351467700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c0a63d0 0x7f334c129760 secure :-1 s=THROTTLE_DONE pgs=163 cs=0 l=1 rx=0x7f334c0a0490 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.315+0000 7f333affd700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f7b0 0x7f332c051c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:39.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.315+0000 7f333affd700 1 -- 172.21.15.196:0/3526958859 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f333c0643a0 con 0x7f334c0a63d0 2020-05-18T21:40:39.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.315+0000 7f3351c68700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f7b0 0x7f332c051c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.315+0000 7f3351c68700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f7b0 0x7f332c051c70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f3348001260 tx=0x7f33480017c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.315+0000 7f3351c68700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f7b0 0x7f332c051c70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f3348001260 tx=0x7f33480017c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.315+0000 7f3351c68700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f7b0 0x7f332c051c70 secure :-1 s=READY pgs=63 cs=0 l=1 rx=0x7f3348001260 tx=0x7f33480017c0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:39.324 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.319+0000 7f3351467700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c0a63d0 0x7f334c129760 secure :-1 s=THROTTLE_DONE pgs=163 cs=0 l=1 rx=0x7f334c0a0490 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.324 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.319+0000 7f333affd700 1 -- 172.21.15.196:0/3526958859 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f333c03e850 con 0x7f334c0a63d0 2020-05-18T21:40:39.436 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.435+0000 7f3352c6a700 1 -- 172.21.15.196:0/3526958859 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f334c0a0620 con 0x7f334c0a63d0 2020-05-18T21:40:39.437 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.439+0000 7f3351467700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c0a63d0 0x7f334c129760 secure :-1 s=THROTTLE_DONE pgs=163 cs=0 l=1 rx=0x7f334c0a0490 tx=0x7f333c007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.438 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.439+0000 7f333affd700 1 -- 172.21.15.196:0/3526958859 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f333c041e70 con 0x7f334c0a63d0 2020-05-18T21:40:39.438 INFO:teuthology.orchestra.run.smithi196.stdout:55834574852 2020-05-18T21:40:39.439 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.439+0000 7f3352c6a700 1 -- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f7b0 msgr2=0x7f332c051c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.440 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.439+0000 7f3352c6a700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f7b0 0x7f332c051c70 secure :-1 s=READY pgs=63 cs=0 l=1 rx=0x7f3348001260 tx=0x7f33480017c0).stop 2020-05-18T21:40:39.440 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.439+0000 7f3352c6a700 1 -- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c0a63d0 msgr2=0x7f334c129760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.440 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.439+0000 7f3352c6a700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c0a63d0 0x7f334c129760 secure :-1 s=READY pgs=163 cs=0 l=1 rx=0x7f334c0a0490 tx=0x7f333c007a80).stop 2020-05-18T21:40:39.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.439+0000 7f3352c6a700 1 -- 172.21.15.196:0/3526958859 shutdown_connections 2020-05-18T21:40:39.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.439+0000 7f3352c6a700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f7b0 0x7f332c051c70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.439+0000 7f3352c6a700 1 --2- 172.21.15.196:0/3526958859 >> v2:172.21.15.80:3300/0 conn(0x7f334c0ab740 0x7f334c148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.439+0000 7f3352c6a700 1 --2- 172.21.15.196:0/3526958859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f334c0a63d0 0x7f334c129760 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.439+0000 7f3352c6a700 1 -- 172.21.15.196:0/3526958859 >> 172.21.15.196:0/3526958859 conn(0x7f334c060e20 msgr2=0x7f334c0aa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:39.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.439+0000 7f3352c6a700 1 -- 172.21.15.196:0/3526958859 shutdown_connections 2020-05-18T21:40:39.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.439+0000 7f3352c6a700 1 -- 172.21.15.196:0/3526958859 wait complete. 2020-05-18T21:40:39.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2020-05-18T21:40:39.450 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.6 2020-05-18T21:40:39.548 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.547+0000 7f92f359e700 1 Processor -- start 2020-05-18T21:40:39.548 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.547+0000 7f92f359e700 1 -- start start 2020-05-18T21:40:39.549 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.547+0000 7f92f359e700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f92f40a2ac0 0x7f92f40a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:39.549 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.547+0000 7f92f359e700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f92f40a7f40 0x7f92f4135560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:39.550 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.547+0000 7f92f359e700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f40a56e0 con 0x7f92f41359c0 2020-05-18T21:40:39.550 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.547+0000 7f92f359e700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f40a4350 con 0x7f92f40a2ac0 2020-05-18T21:40:39.550 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.547+0000 7f92f359e700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f4134ed0 con 0x7f92f40a7f40 2020-05-18T21:40:39.550 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92e9d9b700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f92f40a7f40 0x7f92f4135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.550 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92e9d9b700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f92f40a7f40 0x7f92f4135560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34724/0 (socket says 172.21.15.196:34724) 2020-05-18T21:40:39.551 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92e9d9b700 1 -- 172.21.15.196:0/500995772 learned_addr learned my addr 172.21.15.196:0/500995772 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:39.551 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92f259c700 1 --2- 172.21.15.196:0/500995772 >> v2:172.21.15.80:3300/0 conn(0x7f92f40a2ac0 0x7f92f40a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.553 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92e9d9b700 1 -- 172.21.15.196:0/500995772 >> v2:172.21.15.80:3300/0 conn(0x7f92f40a2ac0 msgr2=0x7f92f40a7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:39.553 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92e9d9b700 1 --2- 172.21.15.196:0/500995772 >> v2:172.21.15.80:3300/0 conn(0x7f92f40a2ac0 0x7f92f40a7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:39.553 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92e9d9b700 1 -- 172.21.15.196:0/500995772 >> v1:172.21.15.80:6789/0 conn(0x7f92f41359c0 legacy=0x7f92f4138420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.554 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92e9d9b700 1 -- 172.21.15.196:0/500995772 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f40a83f0 con 0x7f92f40a7f40 2020-05-18T21:40:39.554 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92e9d9b700 1 --2- 172.21.15.196:0/500995772 >> v2:172.21.15.196:3300/0 conn(0x7f92f40a7f40 0x7f92f4135560 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f92dc002f70 tx=0x7f92dc007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.554 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92e9d9b700 1 --2- 172.21.15.196:0/500995772 >> v2:172.21.15.196:3300/0 conn(0x7f92f40a7f40 0x7f92f4135560 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f92dc002f70 tx=0x7f92dc007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.554 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92e9d9b700 1 --2- 172.21.15.196:0/500995772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92f40a7f40 0x7f92f4135560 secure :-1 s=READY pgs=164 cs=0 l=1 rx=0x7f92dc002f70 tx=0x7f92dc007a80).ready entity=mon.1 client_cookie=f1d5650e1a692a3e server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:39.554 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92e9d9b700 1 --2- 172.21.15.196:0/500995772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92f40a7f40 0x7f92f4135560 secure :-1 s=THROTTLE_DONE pgs=164 cs=0 l=1 rx=0x7f92dc002f70 tx=0x7f92dc007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.555 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92f1d9b700 1 -- 172.21.15.196:0/500995772 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92dc0116b0 con 0x7f92f40a7f40 2020-05-18T21:40:39.555 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92e9d9b700 1 --2- 172.21.15.196:0/500995772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92f40a7f40 0x7f92f4135560 secure :-1 s=THROTTLE_DONE pgs=164 cs=0 l=1 rx=0x7f92dc002f70 tx=0x7f92dc007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.555 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92e9d9b700 1 --2- 172.21.15.196:0/500995772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92f40a7f40 0x7f92f4135560 secure :-1 s=THROTTLE_DONE pgs=164 cs=0 l=1 rx=0x7f92dc002f70 tx=0x7f92dc007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.555 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92f1d9b700 1 -- 172.21.15.196:0/500995772 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc011810 con 0x7f92f40a7f40 2020-05-18T21:40:39.555 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92f1d9b700 1 -- 172.21.15.196:0/500995772 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92dc011c30 con 0x7f92f40a7f40 2020-05-18T21:40:39.555 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92f359e700 1 -- 172.21.15.196:0/500995772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92f40a7f40 msgr2=0x7f92f4135560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.556 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92f359e700 1 --2- 172.21.15.196:0/500995772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92f40a7f40 0x7f92f4135560 secure :-1 s=READY pgs=164 cs=0 l=1 rx=0x7f92dc002f70 tx=0x7f92dc007a80).stop 2020-05-18T21:40:39.556 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92f359e700 1 -- 172.21.15.196:0/500995772 shutdown_connections 2020-05-18T21:40:39.556 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92f359e700 1 --2- 172.21.15.196:0/500995772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92f40a7f40 0x7f92f4135560 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.556 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92f359e700 1 --2- 172.21.15.196:0/500995772 >> v2:172.21.15.80:3300/0 conn(0x7f92f40a2ac0 0x7f92f40a7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:39.556 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92f359e700 1 -- 172.21.15.196:0/500995772 >> 172.21.15.196:0/500995772 conn(0x7f92f4069ac0 msgr2=0x7f92f4069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:39.557 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92f359e700 1 -- 172.21.15.196:0/500995772 shutdown_connections 2020-05-18T21:40:39.557 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.551+0000 7f92f359e700 1 -- 172.21.15.196:0/500995772 wait complete. 2020-05-18T21:40:39.559 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f359e700 1 Processor -- start 2020-05-18T21:40:39.559 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f359e700 1 -- start start 2020-05-18T21:40:39.560 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f359e700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f92f40a2ac0 0x7f92f4138230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:39.560 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f359e700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f92f41359c0 0x7f92f409c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:39.560 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f359e700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f409afd0 con 0x7f92f40a7f40 2020-05-18T21:40:39.560 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f359e700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f4138c10 con 0x7f92f41359c0 2020-05-18T21:40:39.560 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f359e700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f4137be0 con 0x7f92f40a2ac0 2020-05-18T21:40:39.561 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f2d9d700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f92f41359c0 0x7f92f409c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.561 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f259c700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f92f40a2ac0 0x7f92f4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.561 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f2d9d700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f92f41359c0 0x7f92f409c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.196:38958/0 (socket says 172.21.15.196:38958) 2020-05-18T21:40:39.561 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92e9d9b700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f92f40a7f40 0x7f92f409a8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.196:39318/0 (socket says 172.21.15.196:39318) 2020-05-18T21:40:39.561 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f259c700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f92f40a2ac0 0x7f92f4138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.196:38560/0 (socket says 172.21.15.196:38560) 2020-05-18T21:40:39.562 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f2d9d700 1 -- 172.21.15.196:0/2071901007 learned_addr learned my addr 172.21.15.196:0/2071901007 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:39.562 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f2d9d700 1 -- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f92f40a2ac0 msgr2=0x7f92f4138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.563 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f2d9d700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f92f40a2ac0 0x7f92f4138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.563 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f2d9d700 1 -- 172.21.15.196:0/2071901007 >> v1:172.21.15.80:6789/0 conn(0x7f92f40a7f40 legacy=0x7f92f409a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.563 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92ebfff700 1 -- 172.21.15.196:0/2071901007 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584532758 0 0) 0x7f92f409afd0 con 0x7f92f40a7f40 2020-05-18T21:40:39.563 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f2d9d700 1 -- 172.21.15.196:0/2071901007 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f4136040 con 0x7f92f41359c0 2020-05-18T21:40:39.563 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f2d9d700 1 --2- 172.21.15.196:0/2071901007 >> v2:172.21.15.80:3300/0 conn(0x7f92f41359c0 0x7f92f409c5d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f92f4069310 tx=0x7f92ec0021f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f2d9d700 1 --2- 172.21.15.196:0/2071901007 >> v2:172.21.15.80:3300/0 conn(0x7f92f41359c0 0x7f92f409c5d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f92f4069310 tx=0x7f92ec0021f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.555+0000 7f92f2d9d700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f41359c0 0x7f92f409c5d0 secure :-1 s=READY pgs=120 cs=0 l=1 rx=0x7f92f4069310 tx=0x7f92ec0021f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:39.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92f2d9d700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f41359c0 0x7f92f409c5d0 secure :-1 s=THROTTLE_DONE pgs=120 cs=0 l=1 rx=0x7f92f4069310 tx=0x7f92ec0021f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92ebfff700 1 -- 172.21.15.196:0/2071901007 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92ec0103d0 con 0x7f92f41359c0 2020-05-18T21:40:39.564 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92f2d9d700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f41359c0 0x7f92f409c5d0 secure :-1 s=THROTTLE_DONE pgs=120 cs=0 l=1 rx=0x7f92f4069310 tx=0x7f92ec0021f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.565 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92ebfff700 1 -- 172.21.15.196:0/2071901007 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ec010530 con 0x7f92f41359c0 2020-05-18T21:40:39.565 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92f359e700 1 -- 172.21.15.196:0/2071901007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f409d9a0 con 0x7f92f41359c0 2020-05-18T21:40:39.565 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92f2d9d700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f41359c0 0x7f92f409c5d0 secure :-1 s=THROTTLE_DONE pgs=120 cs=0 l=1 rx=0x7f92f4069310 tx=0x7f92ec0021f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.565 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92ebfff700 1 -- 172.21.15.196:0/2071901007 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92ec012360 con 0x7f92f41359c0 2020-05-18T21:40:39.565 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92f359e700 1 -- 172.21.15.196:0/2071901007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f4137060 con 0x7f92f41359c0 2020-05-18T21:40:39.566 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92f2d9d700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f41359c0 0x7f92f409c5d0 secure :-1 s=THROTTLE_DONE pgs=120 cs=0 l=1 rx=0x7f92f4069310 tx=0x7f92ec0021f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.566 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92f359e700 1 -- 172.21.15.196:0/2071901007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f40569e0 con 0x7f92f41359c0 2020-05-18T21:40:39.566 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92ebfff700 1 -- 172.21.15.196:0/2071901007 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f92ec0068a0 con 0x7f92f41359c0 2020-05-18T21:40:39.566 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92f2d9d700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f41359c0 0x7f92f409c5d0 secure :-1 s=THROTTLE_DONE pgs=120 cs=0 l=1 rx=0x7f92f4069310 tx=0x7f92ec0021f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.567 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92ebfff700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f92d804f7c0 0x7f92d8051c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:39.568 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92ebfff700 1 -- 172.21.15.196:0/2071901007 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f92ec064060 con 0x7f92f41359c0 2020-05-18T21:40:39.568 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92f259c700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f92d804f7c0 0x7f92d8051c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:39.568 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92f259c700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f92d804f7c0 0x7f92d8051c80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f92e4002f70 tx=0x7f92e400d8f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.568 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92f259c700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f92d804f7c0 0x7f92d8051c80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f92e4002f70 tx=0x7f92e400d8f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.568 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.559+0000 7f92f259c700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f92d804f7c0 0x7f92d8051c80 secure :-1 s=READY pgs=64 cs=0 l=1 rx=0x7f92e4002f70 tx=0x7f92e400d8f0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:39.569 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.567+0000 7f92f2d9d700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f41359c0 0x7f92f409c5d0 secure :-1 s=THROTTLE_DONE pgs=120 cs=0 l=1 rx=0x7f92f4069310 tx=0x7f92ec0021f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.569 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.567+0000 7f92ebfff700 1 -- 172.21.15.196:0/2071901007 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f92ec03e390 con 0x7f92f41359c0 2020-05-18T21:40:39.685 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.687+0000 7f92f359e700 1 -- 172.21.15.196:0/2071901007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f92f4067a10 con 0x7f92f41359c0 2020-05-18T21:40:39.686 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.687+0000 7f92f2d9d700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f41359c0 0x7f92f409c5d0 secure :-1 s=THROTTLE_DONE pgs=120 cs=0 l=1 rx=0x7f92f4069310 tx=0x7f92ec0021f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:39.686 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.687+0000 7f92ebfff700 1 -- 172.21.15.196:0/2071901007 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f92ec0419b0 con 0x7f92f41359c0 2020-05-18T21:40:39.687 INFO:teuthology.orchestra.run.smithi196.stdout:55834574851 2020-05-18T21:40:39.690 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.687+0000 7f92f359e700 1 -- 172.21.15.196:0/2071901007 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f92d804f7c0 msgr2=0x7f92d8051c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.690 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.687+0000 7f92f359e700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f92d804f7c0 0x7f92d8051c80 secure :-1 s=READY pgs=64 cs=0 l=1 rx=0x7f92e4002f70 tx=0x7f92e400d8f0).stop 2020-05-18T21:40:39.690 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.691+0000 7f92f359e700 1 -- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f41359c0 msgr2=0x7f92f409c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:39.690 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.691+0000 7f92f359e700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f41359c0 0x7f92f409c5d0 secure :-1 s=READY pgs=120 cs=0 l=1 rx=0x7f92f4069310 tx=0x7f92ec0021f0).stop 2020-05-18T21:40:39.690 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.691+0000 7f92f359e700 1 -- 172.21.15.196:0/2071901007 shutdown_connections 2020-05-18T21:40:39.691 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.691+0000 7f92f359e700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f41359c0 0x7f92f409c5d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.691 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.691+0000 7f92f359e700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f92d804f7c0 0x7f92d8051c80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.691 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.691+0000 7f92f359e700 1 --2- 172.21.15.196:0/2071901007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f92f40a2ac0 0x7f92f4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:39.691 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.691+0000 7f92f359e700 1 -- 172.21.15.196:0/2071901007 >> 172.21.15.196:0/2071901007 conn(0x7f92f4069ac0 msgr2=0x7f92f409e840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:39.691 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.691+0000 7f92f359e700 1 -- 172.21.15.196:0/2071901007 shutdown_connections 2020-05-18T21:40:39.692 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:39.691+0000 7f92f359e700 1 -- 172.21.15.196:0/2071901007 wait complete. 2020-05-18T21:40:39.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.6 2020-05-18T21:40:40.707 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.6 2020-05-18T21:40:40.819 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060f59e700 1 Processor -- start 2020-05-18T21:40:40.819 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060f59e700 1 -- start start 2020-05-18T21:40:40.822 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060f59e700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f06100a2ac0 0x7f06100a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:40.822 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060f59e700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06100a7f40 0x7f0610135560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:40.822 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060f59e700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f06101359c0 0x7f0610138420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:40.822 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060f59e700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06100a4350 con 0x7f06101359c0 2020-05-18T21:40:40.822 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060f59e700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0610134ed0 con 0x7f06100a2ac0 2020-05-18T21:40:40.823 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060f59e700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100a3650 con 0x7f06100a7f40 2020-05-18T21:40:40.823 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f06100a2ac0 0x7f06100a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:40.823 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f06100a2ac0 0x7f06100a7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34740/0 (socket says 172.21.15.196:34740) 2020-05-18T21:40:40.823 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 -- 172.21.15.196:0/2499185382 learned_addr learned my addr 172.21.15.196:0/2499185382 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:40.823 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060ed9d700 1 --2- 172.21.15.196:0/2499185382 >> v2:172.21.15.80:3300/0 conn(0x7f06101359c0 0x7f0610138420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:40.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060dd9b700 1 --2- 172.21.15.196:0/2499185382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06100a7f40 0x7f0610135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:40.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 -- 172.21.15.196:0/2499185382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06100a7f40 msgr2=0x7f0610135560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:40.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 --2- 172.21.15.196:0/2499185382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06100a7f40 0x7f0610135560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:40.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 -- 172.21.15.196:0/2499185382 >> v2:172.21.15.80:3300/0 conn(0x7f06101359c0 msgr2=0x7f0610138420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:40.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 --2- 172.21.15.196:0/2499185382 >> v2:172.21.15.80:3300/0 conn(0x7f06101359c0 0x7f0610138420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:40.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 -- 172.21.15.196:0/2499185382 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06100a56e0 con 0x7f06100a2ac0 2020-05-18T21:40:40.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060ed9d700 1 --2- 172.21.15.196:0/2499185382 >> v2:172.21.15.80:3300/0 conn(0x7f06101359c0 0x7f0610138420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_reply_more state changed! 2020-05-18T21:40:40.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 --2- 172.21.15.196:0/2499185382 >> v2:172.21.15.196:3300/0 conn(0x7f06100a2ac0 0x7f06100a7ae0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.826 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 --2- 172.21.15.196:0/2499185382 >> v2:172.21.15.196:3300/0 conn(0x7f06100a2ac0 0x7f06100a7ae0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.826 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 --2- 172.21.15.196:0/2499185382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100a7ae0 secure :-1 s=READY pgs=165 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).ready entity=mon.1 client_cookie=94a25aaf8b58693c server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:40.826 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 --2- 172.21.15.196:0/2499185382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100a7ae0 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.826 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 --2- 172.21.15.196:0/2499185382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100a7ae0 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.826 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060d59a700 1 -- 172.21.15.196:0/2499185382 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0604011760 con 0x7f06100a2ac0 2020-05-18T21:40:40.827 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060e59c700 1 --2- 172.21.15.196:0/2499185382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100a7ae0 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.827 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060d59a700 1 -- 172.21.15.196:0/2499185382 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06040118c0 con 0x7f06100a2ac0 2020-05-18T21:40:40.827 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060d59a700 1 -- 172.21.15.196:0/2499185382 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0604011ce0 con 0x7f06100a2ac0 2020-05-18T21:40:40.827 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060f59e700 1 -- 172.21.15.196:0/2499185382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 msgr2=0x7f06100a7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:40.827 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.819+0000 7f060f59e700 1 --2- 172.21.15.196:0/2499185382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100a7ae0 secure :-1 s=READY pgs=165 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).stop 2020-05-18T21:40:40.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 -- 172.21.15.196:0/2499185382 shutdown_connections 2020-05-18T21:40:40.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 --2- 172.21.15.196:0/2499185382 >> v2:172.21.15.80:3300/0 conn(0x7f06101359c0 0x7f0610138420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:40.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 --2- 172.21.15.196:0/2499185382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06100a7f40 0x7f0610135560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:40.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 --2- 172.21.15.196:0/2499185382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100a7ae0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:40.828 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 -- 172.21.15.196:0/2499185382 >> 172.21.15.196:0/2499185382 conn(0x7f0610069ac0 msgr2=0x7f0610069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:40.829 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 -- 172.21.15.196:0/2499185382 shutdown_connections 2020-05-18T21:40:40.829 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 -- 172.21.15.196:0/2499185382 wait complete. 2020-05-18T21:40:40.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 Processor -- start 2020-05-18T21:40:40.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 -- start start 2020-05-18T21:40:40.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f06100a2ac0 0x7f06100667c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:40.833 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f06100a7f40 0x7f061012d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:40.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06101359c0 0x7f061012dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:40.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0610066f60 con 0x7f06100a7f40 2020-05-18T21:40:40.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0610130920 con 0x7f06100a2ac0 2020-05-18T21:40:40.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0610130bc0 con 0x7f06101359c0 2020-05-18T21:40:40.834 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f06100a2ac0 0x7f06100667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:40.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f06100a2ac0 0x7f06100667c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34744/0 (socket says 172.21.15.196:34744) 2020-05-18T21:40:40.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 -- 172.21.15.196:0/3702183067 learned_addr learned my addr 172.21.15.196:0/3702183067 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:40.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060dd9b700 1 --2- 172.21.15.196:0/3702183067 >> v2:172.21.15.80:3300/0 conn(0x7f06100a7f40 0x7f061012d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:40.835 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060ed9d700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06101359c0 0x7f061012dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:40.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 -- 172.21.15.196:0/3702183067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06101359c0 msgr2=0x7f061012dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:40.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06101359c0 0x7f061012dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:40.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 -- 172.21.15.196:0/3702183067 >> v2:172.21.15.80:3300/0 conn(0x7f06100a7f40 msgr2=0x7f061012d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:40.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 --2- 172.21.15.196:0/3702183067 >> v2:172.21.15.80:3300/0 conn(0x7f06100a7f40 0x7f061012d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:40.836 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 -- 172.21.15.196:0/3702183067 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0610131060 con 0x7f06100a2ac0 2020-05-18T21:40:40.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060dd9b700 1 --2- 172.21.15.196:0/3702183067 >> v2:172.21.15.80:3300/0 conn(0x7f06100a7f40 0x7f061012d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request state changed! 2020-05-18T21:40:40.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060ed9d700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06101359c0 0x7f061012dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request state changed! 2020-05-18T21:40:40.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 --2- 172.21.15.196:0/3702183067 >> v2:172.21.15.196:3300/0 conn(0x7f06100a2ac0 0x7f06100667c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 --2- 172.21.15.196:0/3702183067 >> v2:172.21.15.196:3300/0 conn(0x7f06100a2ac0 0x7f06100667c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.837 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100667c0 secure :-1 s=READY pgs=166 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:40.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100667c0 secure :-1 s=THROTTLE_DONE pgs=166 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100667c0 secure :-1 s=THROTTLE_DONE pgs=166 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f05ff7fe700 1 -- 172.21.15.196:0/3702183067 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0604011e50 con 0x7f06100a2ac0 2020-05-18T21:40:40.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100667c0 secure :-1 s=THROTTLE_DONE pgs=166 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.838 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f05ff7fe700 1 -- 172.21.15.196:0/3702183067 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0604007360 con 0x7f06100a2ac0 2020-05-18T21:40:40.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f05ff7fe700 1 -- 172.21.15.196:0/3702183067 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0604005700 con 0x7f06100a2ac0 2020-05-18T21:40:40.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 -- 172.21.15.196:0/3702183067 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0610136040 con 0x7f06100a2ac0 2020-05-18T21:40:40.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 -- 172.21.15.196:0/3702183067 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061012ec60 con 0x7f06100a2ac0 2020-05-18T21:40:40.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060f59e700 1 -- 172.21.15.196:0/3702183067 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0610130dc0 con 0x7f06100a2ac0 2020-05-18T21:40:40.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.823+0000 7f060e59c700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100667c0 secure :-1 s=THROTTLE_DONE pgs=166 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.840 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.827+0000 7f05ff7fe700 1 -- 172.21.15.196:0/3702183067 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f0604005860 con 0x7f06100a2ac0 2020-05-18T21:40:40.840 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.827+0000 7f060e59c700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100667c0 secure :-1 s=THROTTLE_DONE pgs=166 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.840 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.827+0000 7f05ff7fe700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f05ec04f930 0x7f05ec051df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:40.840 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.827+0000 7f05ff7fe700 1 -- 172.21.15.196:0/3702183067 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f06040644f0 con 0x7f06100a2ac0 2020-05-18T21:40:40.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.827+0000 7f060dd9b700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f05ec04f930 0x7f05ec051df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:40.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.827+0000 7f060dd9b700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f05ec04f930 0x7f05ec051df0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0600002f70 tx=0x7f0600008e60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.827+0000 7f060dd9b700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f05ec04f930 0x7f05ec051df0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0600002f70 tx=0x7f0600008e60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.827+0000 7f060dd9b700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f05ec04f930 0x7f05ec051df0 secure :-1 s=READY pgs=66 cs=0 l=1 rx=0x7f0600002f70 tx=0x7f0600008e60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:40.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.831+0000 7f060e59c700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100667c0 secure :-1 s=THROTTLE_DONE pgs=166 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.842 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.831+0000 7f05ff7fe700 1 -- 172.21.15.196:0/3702183067 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f0610130dc0 con 0x7f06100a2ac0 2020-05-18T21:40:40.945 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.943+0000 7f060f59e700 1 -- 172.21.15.196:0/3702183067 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0610130d90 con 0x7f06100a2ac0 2020-05-18T21:40:40.946 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f060e59c700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100667c0 secure :-1 s=THROTTLE_DONE pgs=166 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:40.946 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f05ff7fe700 1 -- 172.21.15.196:0/3702183067 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f060403e9a0 con 0x7f06100a2ac0 2020-05-18T21:40:40.946 INFO:teuthology.orchestra.run.smithi196.stdout:55834574853 2020-05-18T21:40:40.947 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f060f59e700 1 -- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f05ec04f930 msgr2=0x7f05ec051df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:40.947 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f060f59e700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f05ec04f930 0x7f05ec051df0 secure :-1 s=READY pgs=66 cs=0 l=1 rx=0x7f0600002f70 tx=0x7f0600008e60).stop 2020-05-18T21:40:40.948 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f060f59e700 1 -- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 msgr2=0x7f06100667c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:40.948 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f060f59e700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100667c0 secure :-1 s=READY pgs=166 cs=0 l=1 rx=0x7f0604002f70 tx=0x7f0604007a80).stop 2020-05-18T21:40:40.948 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f060f59e700 1 -- 172.21.15.196:0/3702183067 shutdown_connections 2020-05-18T21:40:40.948 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f060f59e700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f05ec04f930 0x7f05ec051df0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:40.948 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f060f59e700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06101359c0 0x7f061012dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:40.949 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f060f59e700 1 --2- 172.21.15.196:0/3702183067 >> v2:172.21.15.80:3300/0 conn(0x7f06100a7f40 0x7f061012d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:40.949 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f060f59e700 1 --2- 172.21.15.196:0/3702183067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06100a2ac0 0x7f06100667c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:40.949 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f060f59e700 1 -- 172.21.15.196:0/3702183067 >> 172.21.15.196:0/3702183067 conn(0x7f0610069ac0 msgr2=0x7f061005eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:40.949 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f060f59e700 1 -- 172.21.15.196:0/3702183067 shutdown_connections 2020-05-18T21:40:40.949 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:40.947+0000 7f060f59e700 1 -- 172.21.15.196:0/3702183067 wait complete. 2020-05-18T21:40:40.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.6 2020-05-18T21:40:40.964 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.7 2020-05-18T21:40:41.063 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f252f3700 1 Processor -- start 2020-05-18T21:40:41.063 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f252f3700 1 -- start start 2020-05-18T21:40:41.066 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f252f3700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6f200a6230 0x7f6f2013dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:41.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f252f3700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6f2013e180 0x7f6f20140570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:41.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f252f3700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f200a4850 con 0x7f6f200a3e40 2020-05-18T21:40:41.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f252f3700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f200a5890 con 0x7f6f2013e180 2020-05-18T21:40:41.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f252f3700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2009dff0 con 0x7f6f200a6230 2020-05-18T21:40:41.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f177fe700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6f200a6230 0x7f6f2013dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f24af2700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6f2013e180 0x7f6f20140570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f177fe700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6f200a6230 0x7f6f2013dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.196:38584/0 (socket says 172.21.15.196:38584) 2020-05-18T21:40:41.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f177fe700 1 -- 172.21.15.196:0/3443629769 learned_addr learned my addr 172.21.15.196:0/3443629769 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:41.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f177fe700 1 -- 172.21.15.196:0/3443629769 >> v2:172.21.15.80:3300/0 conn(0x7f6f2013e180 msgr2=0x7f6f20140570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:41.069 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f177fe700 1 --2- 172.21.15.196:0/3443629769 >> v2:172.21.15.80:3300/0 conn(0x7f6f2013e180 0x7f6f20140570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:41.069 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f177fe700 1 -- 172.21.15.196:0/3443629769 >> v1:172.21.15.80:6789/0 conn(0x7f6f200a3e40 legacy=0x7f6f200a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.069 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f177fe700 1 -- 172.21.15.196:0/3443629769 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f200a5240 con 0x7f6f200a6230 2020-05-18T21:40:41.069 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f177fe700 1 --2- 172.21.15.196:0/3443629769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6f200a6230 0x7f6f2013dd20 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f6f08002f70 tx=0x7f6f08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.069 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f177fe700 1 --2- 172.21.15.196:0/3443629769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6f200a6230 0x7f6f2013dd20 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f6f08002f70 tx=0x7f6f08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.070 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.063+0000 7f6f177fe700 1 --2- 172.21.15.196:0/3443629769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6f200a6230 0x7f6f2013dd20 secure :-1 s=READY pgs=29 cs=0 l=1 rx=0x7f6f08002f70 tx=0x7f6f08007a80).ready entity=mon.2 client_cookie=dc7ef8d8b60e6c64 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:41.070 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f177fe700 1 --2- 172.21.15.196:0/3443629769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6f200a6230 0x7f6f2013dd20 secure :-1 s=THROTTLE_DONE pgs=29 cs=0 l=1 rx=0x7f6f08002f70 tx=0x7f6f08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.070 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f1f7fe700 1 -- 172.21.15.196:0/3443629769 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f080116b0 con 0x7f6f200a6230 2020-05-18T21:40:41.070 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f177fe700 1 --2- 172.21.15.196:0/3443629769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6f200a6230 0x7f6f2013dd20 secure :-1 s=THROTTLE_DONE pgs=29 cs=0 l=1 rx=0x7f6f08002f70 tx=0x7f6f08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.070 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f1f7fe700 1 -- 172.21.15.196:0/3443629769 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f08011810 con 0x7f6f200a6230 2020-05-18T21:40:41.071 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f177fe700 1 --2- 172.21.15.196:0/3443629769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6f200a6230 0x7f6f2013dd20 secure :-1 s=THROTTLE_DONE pgs=29 cs=0 l=1 rx=0x7f6f08002f70 tx=0x7f6f08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.071 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f1f7fe700 1 -- 172.21.15.196:0/3443629769 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f08011c30 con 0x7f6f200a6230 2020-05-18T21:40:41.071 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f252f3700 1 -- 172.21.15.196:0/3443629769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6f200a6230 msgr2=0x7f6f2013dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.071 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f252f3700 1 --2- 172.21.15.196:0/3443629769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6f200a6230 0x7f6f2013dd20 secure :-1 s=READY pgs=29 cs=0 l=1 rx=0x7f6f08002f70 tx=0x7f6f08007a80).stop 2020-05-18T21:40:41.071 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f252f3700 1 -- 172.21.15.196:0/3443629769 shutdown_connections 2020-05-18T21:40:41.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f252f3700 1 --2- 172.21.15.196:0/3443629769 >> v2:172.21.15.80:3300/0 conn(0x7f6f2013e180 0x7f6f20140570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:41.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f252f3700 1 --2- 172.21.15.196:0/3443629769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6f200a6230 0x7f6f2013dd20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f252f3700 1 -- 172.21.15.196:0/3443629769 >> 172.21.15.196:0/3443629769 conn(0x7f6f2005a7c0 msgr2=0x7f6f2005abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:41.072 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f252f3700 1 -- 172.21.15.196:0/3443629769 shutdown_connections 2020-05-18T21:40:41.073 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f252f3700 1 -- 172.21.15.196:0/3443629769 wait complete. 2020-05-18T21:40:41.076 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f252f3700 1 Processor -- start 2020-05-18T21:40:41.077 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.067+0000 7f6f252f3700 1 -- start start 2020-05-18T21:40:41.077 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f252f3700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6f200a3e40 0x7f6f2013f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:41.077 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f252f3700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6f200a6230 0x7f6f200a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:41.077 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f252f3700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2013ed20 con 0x7f6f2013e180 2020-05-18T21:40:41.077 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f252f3700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2013ef80 con 0x7f6f200a6230 2020-05-18T21:40:41.078 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f252f3700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2013f220 con 0x7f6f200a3e40 2020-05-18T21:40:41.078 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6f200a3e40 0x7f6f2013f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.079 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6f200a3e40 0x7f6f2013f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34758/0 (socket says 172.21.15.196:34758) 2020-05-18T21:40:41.079 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 -- 172.21.15.196:0/3847373753 learned_addr learned my addr 172.21.15.196:0/3847373753 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:41.079 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f177fe700 1 --2- 172.21.15.196:0/3847373753 >> v2:172.21.15.80:3300/0 conn(0x7f6f200a6230 0x7f6f200a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.080 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 -- 172.21.15.196:0/3847373753 >> v2:172.21.15.80:3300/0 conn(0x7f6f200a6230 msgr2=0x7f6f200a8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.080 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 --2- 172.21.15.196:0/3847373753 >> v2:172.21.15.80:3300/0 conn(0x7f6f200a6230 0x7f6f200a8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.080 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 -- 172.21.15.196:0/3847373753 >> v1:172.21.15.80:6789/0 conn(0x7f6f2013e180 legacy=0x7f6f20140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.080 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 -- 172.21.15.196:0/3847373753 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f200a0f10 con 0x7f6f200a3e40 2020-05-18T21:40:41.081 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 --2- 172.21.15.196:0/3847373753 >> v2:172.21.15.196:3300/0 conn(0x7f6f200a3e40 0x7f6f2013f6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f6f2009dc10 tx=0x7f6f1000c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.081 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 --2- 172.21.15.196:0/3847373753 >> v2:172.21.15.196:3300/0 conn(0x7f6f200a3e40 0x7f6f2013f6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f6f2009dc10 tx=0x7f6f1000c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.081 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f200a3e40 0x7f6f2013f6c0 secure :-1 s=READY pgs=167 cs=0 l=1 rx=0x7f6f2009dc10 tx=0x7f6f1000c4a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:41.082 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f200a3e40 0x7f6f2013f6c0 secure :-1 s=THROTTLE_DONE pgs=167 cs=0 l=1 rx=0x7f6f2009dc10 tx=0x7f6f1000c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.082 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f200a3e40 0x7f6f2013f6c0 secure :-1 s=THROTTLE_DONE pgs=167 cs=0 l=1 rx=0x7f6f2009dc10 tx=0x7f6f1000c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.082 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1dffb700 1 -- 172.21.15.196:0/3847373753 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f1000cc60 con 0x7f6f200a3e40 2020-05-18T21:40:41.083 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f200a3e40 0x7f6f2013f6c0 secure :-1 s=THROTTLE_DONE pgs=167 cs=0 l=1 rx=0x7f6f2009dc10 tx=0x7f6f1000c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.083 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1dffb700 1 -- 172.21.15.196:0/3847373753 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1000cdc0 con 0x7f6f200a3e40 2020-05-18T21:40:41.083 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f252f3700 1 -- 172.21.15.196:0/3847373753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2009f250 con 0x7f6f200a3e40 2020-05-18T21:40:41.083 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1dffb700 1 -- 172.21.15.196:0/3847373753 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f100111b0 con 0x7f6f200a3e40 2020-05-18T21:40:41.084 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f252f3700 1 -- 172.21.15.196:0/3847373753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2009f700 con 0x7f6f200a3e40 2020-05-18T21:40:41.084 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f200a3e40 0x7f6f2013f6c0 secure :-1 s=THROTTLE_DONE pgs=167 cs=0 l=1 rx=0x7f6f2009dc10 tx=0x7f6f1000c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.084 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1dffb700 1 -- 172.21.15.196:0/3847373753 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f6f10006520 con 0x7f6f200a3e40 2020-05-18T21:40:41.085 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1ffff700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f200a3e40 0x7f6f2013f6c0 secure :-1 s=THROTTLE_DONE pgs=167 cs=0 l=1 rx=0x7f6f2009dc10 tx=0x7f6f1000c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.085 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f252f3700 1 -- 172.21.15.196:0/3847373753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f200569e0 con 0x7f6f200a3e40 2020-05-18T21:40:41.085 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1dffb700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6f0404fa20 0x7f6f04051ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:41.086 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f1dffb700 1 -- 172.21.15.196:0/3847373753 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6f100637b0 con 0x7f6f200a3e40 2020-05-18T21:40:41.086 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.071+0000 7f6f177fe700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6f0404fa20 0x7f6f04051ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.087 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.075+0000 7f6f177fe700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6f0404fa20 0x7f6f04051ee0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f6f08002f70 tx=0x7f6f08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.087 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.075+0000 7f6f177fe700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6f0404fa20 0x7f6f04051ee0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f6f08002f70 tx=0x7f6f08007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.087 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.075+0000 7f6f177fe700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6f0404fa20 0x7f6f04051ee0 secure :-1 s=READY pgs=67 cs=0 l=1 rx=0x7f6f08002f70 tx=0x7f6f08007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:41.088 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.083+0000 7f6f1ffff700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f200a3e40 0x7f6f2013f6c0 secure :-1 s=THROTTLE_DONE pgs=167 cs=0 l=1 rx=0x7f6f2009dc10 tx=0x7f6f1000c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.088 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.083+0000 7f6f1dffb700 1 -- 172.21.15.196:0/3847373753 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f6f1003db40 con 0x7f6f200a3e40 2020-05-18T21:40:41.197 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.195+0000 7f6f252f3700 1 -- 172.21.15.196:0/3847373753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f6f20067330 con 0x7f6f200a3e40 2020-05-18T21:40:41.198 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.199+0000 7f6f1ffff700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f200a3e40 0x7f6f2013f6c0 secure :-1 s=THROTTLE_DONE pgs=167 cs=0 l=1 rx=0x7f6f2009dc10 tx=0x7f6f1000c4a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.198 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.199+0000 7f6f1dffb700 1 -- 172.21.15.196:0/3847373753 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6f10041160 con 0x7f6f200a3e40 2020-05-18T21:40:41.198 INFO:teuthology.orchestra.run.smithi196.stdout:55834574853 2020-05-18T21:40:41.199 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.199+0000 7f6f252f3700 1 -- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6f0404fa20 msgr2=0x7f6f04051ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.199 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.199+0000 7f6f252f3700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6f0404fa20 0x7f6f04051ee0 secure :-1 s=READY pgs=67 cs=0 l=1 rx=0x7f6f08002f70 tx=0x7f6f08007a80).stop 2020-05-18T21:40:41.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.199+0000 7f6f252f3700 1 -- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f200a3e40 msgr2=0x7f6f2013f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.199+0000 7f6f252f3700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f200a3e40 0x7f6f2013f6c0 secure :-1 s=READY pgs=167 cs=0 l=1 rx=0x7f6f2009dc10 tx=0x7f6f1000c4a0).stop 2020-05-18T21:40:41.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.199+0000 7f6f252f3700 1 -- 172.21.15.196:0/3847373753 shutdown_connections 2020-05-18T21:40:41.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.199+0000 7f6f252f3700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f6f0404fa20 0x7f6f04051ee0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.199+0000 7f6f252f3700 1 --2- 172.21.15.196:0/3847373753 >> v2:172.21.15.80:3300/0 conn(0x7f6f200a6230 0x7f6f200a8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.199+0000 7f6f252f3700 1 --2- 172.21.15.196:0/3847373753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f200a3e40 0x7f6f2013f6c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.199+0000 7f6f252f3700 1 -- 172.21.15.196:0/3847373753 >> 172.21.15.196:0/3847373753 conn(0x7f6f2005a7c0 msgr2=0x7f6f2005eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:41.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.199+0000 7f6f252f3700 1 -- 172.21.15.196:0/3847373753 shutdown_connections 2020-05-18T21:40:41.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.199+0000 7f6f252f3700 1 -- 172.21.15.196:0/3847373753 wait complete. 2020-05-18T21:40:41.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2020-05-18T21:40:41.216 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-05-18T21:40:41.216 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-05-18T21:40:41.319 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5691ef700 1 Processor -- start 2020-05-18T21:40:41.319 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5691ef700 1 -- start start 2020-05-18T21:40:41.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5691ef700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5640a7220 0x7ff5640a3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:41.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5691ef700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff5640a3dd0 0x7ff564140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:41.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5691ef700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff564140590 0x7ff5640a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:41.322 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5691ef700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5640a3580 con 0x7ff564140590 2020-05-18T21:40:41.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5691ef700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff56409e0b0 con 0x7ff5640a3dd0 2020-05-18T21:40:41.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5691ef700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5640a20f0 con 0x7ff5640a7220 2020-05-18T21:40:41.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff5640a3dd0 0x7ff564140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff5640a3dd0 0x7ff564140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34766/0 (socket says 172.21.15.196:34766) 2020-05-18T21:40:41.323 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 -- 172.21.15.196:0/2629135968 learned_addr learned my addr 172.21.15.196:0/2629135968 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:41.324 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5689ee700 1 --2- 172.21.15.196:0/2629135968 >> v2:172.21.15.80:3300/0 conn(0x7ff564140590 0x7ff5640a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.324 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff563fff700 1 --2- 172.21.15.196:0/2629135968 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5640a7220 0x7ff5640a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.324 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 -- 172.21.15.196:0/2629135968 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5640a7220 msgr2=0x7ff5640a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:41.324 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 --2- 172.21.15.196:0/2629135968 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5640a7220 0x7ff5640a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:41.324 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 -- 172.21.15.196:0/2629135968 >> v2:172.21.15.80:3300/0 conn(0x7ff564140590 msgr2=0x7ff5640a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:41.324 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 --2- 172.21.15.196:0/2629135968 >> v2:172.21.15.80:3300/0 conn(0x7ff564140590 0x7ff5640a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:41.325 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 -- 172.21.15.196:0/2629135968 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5640a6a60 con 0x7ff5640a3dd0 2020-05-18T21:40:41.325 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff563fff700 1 --2- 172.21.15.196:0/2629135968 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5640a7220 0x7ff5640a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request state changed! 2020-05-18T21:40:41.325 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5689ee700 1 --2- 172.21.15.196:0/2629135968 >> v2:172.21.15.80:3300/0 conn(0x7ff564140590 0x7ff5640a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request state changed! 2020-05-18T21:40:41.325 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 --2- 172.21.15.196:0/2629135968 >> v2:172.21.15.196:3300/0 conn(0x7ff5640a3dd0 0x7ff564140130 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7ff5540032f0 tx=0x7ff554007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.325 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 --2- 172.21.15.196:0/2629135968 >> v2:172.21.15.196:3300/0 conn(0x7ff5640a3dd0 0x7ff564140130 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7ff5540032f0 tx=0x7ff554007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.326 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 --2- 172.21.15.196:0/2629135968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a3dd0 0x7ff564140130 secure :-1 s=READY pgs=168 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).ready entity=mon.1 client_cookie=1a2610610962bb83 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:41.326 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 --2- 172.21.15.196:0/2629135968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a3dd0 0x7ff564140130 secure :-1 s=THROTTLE_DONE pgs=168 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.326 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff562ffd700 1 -- 172.21.15.196:0/2629135968 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff554011dc0 con 0x7ff5640a3dd0 2020-05-18T21:40:41.326 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 --2- 172.21.15.196:0/2629135968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a3dd0 0x7ff564140130 secure :-1 s=THROTTLE_DONE pgs=168 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.326 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff562ffd700 1 -- 172.21.15.196:0/2629135968 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55400a180 con 0x7ff5640a3dd0 2020-05-18T21:40:41.327 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5637fe700 1 --2- 172.21.15.196:0/2629135968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a3dd0 0x7ff564140130 secure :-1 s=THROTTLE_DONE pgs=168 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.327 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff562ffd700 1 -- 172.21.15.196:0/2629135968 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff55400a5a0 con 0x7ff5640a3dd0 2020-05-18T21:40:41.327 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5691ef700 1 -- 172.21.15.196:0/2629135968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a3dd0 msgr2=0x7ff564140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.327 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.319+0000 7ff5691ef700 1 --2- 172.21.15.196:0/2629135968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a3dd0 0x7ff564140130 secure :-1 s=READY pgs=168 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).stop 2020-05-18T21:40:41.327 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.323+0000 7ff5691ef700 1 -- 172.21.15.196:0/2629135968 shutdown_connections 2020-05-18T21:40:41.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.323+0000 7ff5691ef700 1 --2- 172.21.15.196:0/2629135968 >> v2:172.21.15.80:3300/0 conn(0x7ff564140590 0x7ff5640a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:41.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.323+0000 7ff5691ef700 1 --2- 172.21.15.196:0/2629135968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a3dd0 0x7ff564140130 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.323+0000 7ff5691ef700 1 --2- 172.21.15.196:0/2629135968 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5640a7220 0x7ff5640a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:41.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.323+0000 7ff5691ef700 1 -- 172.21.15.196:0/2629135968 >> 172.21.15.196:0/2629135968 conn(0x7ff56405a7c0 msgr2=0x7ff56405abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:41.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.323+0000 7ff5691ef700 1 -- 172.21.15.196:0/2629135968 shutdown_connections 2020-05-18T21:40:41.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.323+0000 7ff5691ef700 1 -- 172.21.15.196:0/2629135968 wait complete. 2020-05-18T21:40:41.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.323+0000 7ff5691ef700 1 Processor -- start 2020-05-18T21:40:41.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.323+0000 7ff5691ef700 1 -- start start 2020-05-18T21:40:41.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5691ef700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff5640a7220 0x7ff56413d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:41.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5691ef700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff564140590 0x7ff56409a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:41.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5691ef700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5640a0c80 con 0x7ff5640a3dd0 2020-05-18T21:40:41.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5691ef700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5640a0f20 con 0x7ff5640a7220 2020-05-18T21:40:41.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5691ef700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5640a11c0 con 0x7ff564140590 2020-05-18T21:40:41.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff5640a7220 0x7ff56413d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff5640a7220 0x7ff56413d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34774/0 (socket says 172.21.15.196:34774) 2020-05-18T21:40:41.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 -- 172.21.15.196:0/1849221234 learned_addr learned my addr 172.21.15.196:0/1849221234 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:41.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5689ee700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff564140590 0x7ff56409a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 -- 172.21.15.196:0/1849221234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff564140590 msgr2=0x7ff56409a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff564140590 0x7ff56409a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 -- 172.21.15.196:0/1849221234 >> v1:172.21.15.80:6789/0 conn(0x7ff5640a3dd0 legacy=0x7ff56414ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 -- 172.21.15.196:0/1849221234 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56413dbb0 con 0x7ff5640a7220 2020-05-18T21:40:41.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 --2- 172.21.15.196:0/1849221234 >> v2:172.21.15.196:3300/0 conn(0x7ff5640a7220 0x7ff56413d3e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 --2- 172.21.15.196:0/1849221234 >> v2:172.21.15.196:3300/0 conn(0x7ff5640a7220 0x7ff56413d3e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a7220 0x7ff56413d3e0 secure :-1 s=READY pgs=169 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:41.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a7220 0x7ff56413d3e0 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5617fa700 1 -- 172.21.15.196:0/1849221234 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff554005c70 con 0x7ff5640a7220 2020-05-18T21:40:41.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a7220 0x7ff56413d3e0 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5617fa700 1 -- 172.21.15.196:0/1849221234 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff554005dd0 con 0x7ff5640a7220 2020-05-18T21:40:41.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a7220 0x7ff56413d3e0 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5691ef700 1 -- 172.21.15.196:0/1849221234 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5640a0900 con 0x7ff5640a7220 2020-05-18T21:40:41.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5617fa700 1 -- 172.21.15.196:0/1849221234 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff55400f7a0 con 0x7ff5640a7220 2020-05-18T21:40:41.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5691ef700 1 -- 172.21.15.196:0/1849221234 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56414d760 con 0x7ff5640a7220 2020-05-18T21:40:41.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a7220 0x7ff56413d3e0 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5691ef700 1 -- 172.21.15.196:0/1849221234 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5640569e0 con 0x7ff5640a7220 2020-05-18T21:40:41.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5617fa700 1 -- 172.21.15.196:0/1849221234 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7ff55400f9e0 con 0x7ff5640a7220 2020-05-18T21:40:41.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5637fe700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a7220 0x7ff56413d3e0 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5617fa700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ff54404f950 0x7ff544051e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:41.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff5617fa700 1 -- 172.21.15.196:0/1849221234 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff554064c60 con 0x7ff5640a7220 2020-05-18T21:40:41.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff563fff700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ff54404f950 0x7ff544051e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.327+0000 7ff563fff700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ff54404f950 0x7ff544051e10 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ff56413fa90 tx=0x7ff558001210).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.331+0000 7ff563fff700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ff54404f950 0x7ff544051e10 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ff56413fa90 tx=0x7ff558001210).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.331+0000 7ff563fff700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ff54404f950 0x7ff544051e10 secure :-1 s=READY pgs=68 cs=0 l=1 rx=0x7ff56413fa90 tx=0x7ff558001210).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:41.339 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.335+0000 7ff5637fe700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a7220 0x7ff56413d3e0 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.339 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.335+0000 7ff5617fa700 1 -- 172.21.15.196:0/1849221234 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7ff55403f110 con 0x7ff5640a7220 2020-05-18T21:40:41.437 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:40:41.440 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:40:41.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.443+0000 7ff5691ef700 1 -- 172.21.15.196:0/1849221234 --> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff56414cb90 con 0x7ff54404f950 2020-05-18T21:40:41.450 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.451+0000 7ff563fff700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ff54404f950 0x7ff544051e10 secure :-1 s=THROTTLE_DONE pgs=68 cs=0 l=1 rx=0x7ff56413fa90 tx=0x7ff558001210).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.450 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.451+0000 7ff5617fa700 1 -- 172.21.15.196:0/1849221234 <== mgr.4105 v2:172.21.15.196:6832/12657 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82205 (secure 0 0 0) 0x7ff55800de50 con 0x7ff54404f950 2020-05-18T21:40:41.451 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:41.452 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.451+0000 7ff5691ef700 1 -- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ff54404f950 msgr2=0x7ff544051e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.452 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.451+0000 7ff5691ef700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ff54404f950 0x7ff544051e10 secure :-1 s=READY pgs=68 cs=0 l=1 rx=0x7ff56413fa90 tx=0x7ff558001210).stop 2020-05-18T21:40:41.452 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.451+0000 7ff5691ef700 1 -- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a7220 msgr2=0x7ff56413d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.453 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.451+0000 7ff5691ef700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a7220 0x7ff56413d3e0 secure :-1 s=READY pgs=169 cs=0 l=1 rx=0x7ff5540032f0 tx=0x7ff554007e00).stop 2020-05-18T21:40:41.453 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.451+0000 7ff5691ef700 1 -- 172.21.15.196:0/1849221234 shutdown_connections 2020-05-18T21:40:41.453 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.451+0000 7ff5691ef700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7ff54404f950 0x7ff544051e10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.453 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.451+0000 7ff5691ef700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff564140590 0x7ff56409a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.454 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.451+0000 7ff5691ef700 1 --2- 172.21.15.196:0/1849221234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5640a7220 0x7ff56413d3e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.454 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.451+0000 7ff5691ef700 1 -- 172.21.15.196:0/1849221234 >> 172.21.15.196:0/1849221234 conn(0x7ff56405a7c0 msgr2=0x7ff56405eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:41.454 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.451+0000 7ff5691ef700 1 -- 172.21.15.196:0/1849221234 shutdown_connections 2020-05-18T21:40:41.454 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.451+0000 7ff5691ef700 1 -- 172.21.15.196:0/1849221234 wait complete. 2020-05-18T21:40:41.455 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2020-05-18T21:40:41.465 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2020-05-18T21:40:39.755522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390528,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584192,"statfs":{"total":773094113280,"available":764502212608,"internally_reserved":8589934592,"allocated":1900544,"data_stored":692516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.850714"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.222712+0000","last_change":"2020-05-18T21:40:32.221870+0000","last_active":"2020-05-18T21:40:32.222712+0000","last_peered":"2020-05-18T21:40:32.222712+0000","last_clean":"2020-05-18T21:40:32.222712+0000","last_became_active":"2020-05-18T21:40:32.221106+0000","last_became_peered":"2020-05-18T21:40:32.221106+0000","last_unstale":"2020-05-18T21:40:32.222712+0000","last_undegraded":"2020-05-18T21:40:32.222712+0000","last_fullsized":"2020-05-18T21:40:32.222712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"17","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.264464+0000","last_change":"2020-05-18T21:40:23.820420+0000","last_active":"2020-05-18T21:40:32.264464+0000","last_peered":"2020-05-18T21:40:32.264464+0000","last_clean":"2020-05-18T21:40:32.264464+0000","last_became_active":"2020-05-18T21:40:23.818790+0000","last_became_peered":"2020-05-18T21:40:23.818790+0000","last_unstale":"2020-05-18T21:40:32.264464+0000","last_undegraded":"2020-05-18T21:40:32.264464+0000","last_fullsized":"2020-05-18T21:40:32.264464+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:20.742680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:20.742680+0000","last_clean_scrub_stamp":"2020-05-18T21:40:20.742680+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.564050+0000","last_change":"2020-05-18T21:40:28.872619+0000","last_active":"2020-05-18T21:40:32.564050+0000","last_peered":"2020-05-18T21:40:32.564050+0000","last_clean":"2020-05-18T21:40:32.564050+0000","last_became_active":"2020-05-18T21:40:28.872176+0000","last_became_peered":"2020-05-18T21:40:28.872176+0000","last_unstale":"2020-05-18T21:40:32.564050+0000","last_undegraded":"2020-05-18T21:40:32.564050+0000","last_fullsized":"2020-05-18T21:40:32.564050+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"22'1","reported_seq":"11","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:33.541062+0000","last_change":"2020-05-18T21:40:31.921020+0000","last_active":"2020-05-18T21:40:33.541062+0000","last_peered":"2020-05-18T21:40:33.541062+0000","last_clean":"2020-05-18T21:40:33.541062+0000","last_became_active":"2020-05-18T21:40:31.920583+0000","last_became_peered":"2020-05-18T21:40:31.920583+0000","last_unstale":"2020-05-18T21:40:33.541062+0000","last_undegraded":"2020-05-18T21:40:33.541062+0000","last_fullsized":"2020-05-18T21:40:33.541062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.230558+0000","last_change":"2020-05-18T21:40:32.230208+0000","last_active":"2020-05-18T21:40:32.230558+0000","last_peered":"2020-05-18T21:40:32.230558+0000","last_clean":"2020-05-18T21:40:32.230558+0000","last_became_active":"2020-05-18T21:40:32.229548+0000","last_became_peered":"2020-05-18T21:40:32.229548+0000","last_unstale":"2020-05-18T21:40:32.230558+0000","last_undegraded":"2020-05-18T21:40:32.230558+0000","last_fullsized":"2020-05-18T21:40:32.230558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.264116+0000","last_change":"2020-05-18T21:40:28.870436+0000","last_active":"2020-05-18T21:40:32.264116+0000","last_peered":"2020-05-18T21:40:32.264116+0000","last_clean":"2020-05-18T21:40:32.264116+0000","last_became_active":"2020-05-18T21:40:28.870101+0000","last_became_peered":"2020-05-18T21:40:28.870101+0000","last_unstale":"2020-05-18T21:40:32.264116+0000","last_undegraded":"2020-05-18T21:40:32.264116+0000","last_fullsized":"2020-05-18T21:40:32.264116+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"10","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.226672+0000","last_change":"2020-05-18T21:40:31.919721+0000","last_active":"2020-05-18T21:40:32.226672+0000","last_peered":"2020-05-18T21:40:32.226672+0000","last_clean":"2020-05-18T21:40:32.226672+0000","last_became_active":"2020-05-18T21:40:31.918670+0000","last_became_peered":"2020-05-18T21:40:31.918670+0000","last_unstale":"2020-05-18T21:40:32.226672+0000","last_undegraded":"2020-05-18T21:40:32.226672+0000","last_fullsized":"2020-05-18T21:40:32.226672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.572101+0000","last_change":"2020-05-18T21:40:32.571510+0000","last_active":"2020-05-18T21:40:32.572101+0000","last_peered":"2020-05-18T21:40:32.572101+0000","last_clean":"2020-05-18T21:40:32.572101+0000","last_became_active":"2020-05-18T21:40:32.570722+0000","last_became_peered":"2020-05-18T21:40:32.570722+0000","last_unstale":"2020-05-18T21:40:32.572101+0000","last_undegraded":"2020-05-18T21:40:32.572101+0000","last_fullsized":"2020-05-18T21:40:32.572101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.219524+0000","last_change":"2020-05-18T21:40:28.873535+0000","last_active":"2020-05-18T21:40:32.219524+0000","last_peered":"2020-05-18T21:40:32.219524+0000","last_clean":"2020-05-18T21:40:32.219524+0000","last_became_active":"2020-05-18T21:40:28.872824+0000","last_became_peered":"2020-05-18T21:40:28.872824+0000","last_unstale":"2020-05-18T21:40:32.219524+0000","last_undegraded":"2020-05-18T21:40:32.219524+0000","last_fullsized":"2020-05-18T21:40:32.219524+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:35.257628+0000","last_change":"2020-05-18T21:40:31.915603+0000","last_active":"2020-05-18T21:40:35.257628+0000","last_peered":"2020-05-18T21:40:35.257628+0000","last_clean":"2020-05-18T21:40:35.257628+0000","last_became_active":"2020-05-18T21:40:31.913596+0000","last_became_peered":"2020-05-18T21:40:31.913596+0000","last_unstale":"2020-05-18T21:40:35.257628+0000","last_undegraded":"2020-05-18T21:40:35.257628+0000","last_fullsized":"2020-05-18T21:40:35.257628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.229937+0000","last_change":"2020-05-18T21:40:32.229478+0000","last_active":"2020-05-18T21:40:32.229937+0000","last_peered":"2020-05-18T21:40:32.229937+0000","last_clean":"2020-05-18T21:40:32.229937+0000","last_became_active":"2020-05-18T21:40:32.229111+0000","last_became_peered":"2020-05-18T21:40:32.229111+0000","last_unstale":"2020-05-18T21:40:32.229937+0000","last_undegraded":"2020-05-18T21:40:32.229937+0000","last_fullsized":"2020-05-18T21:40:32.229937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.564049+0000","last_change":"2020-05-18T21:40:28.872945+0000","last_active":"2020-05-18T21:40:32.564049+0000","last_peered":"2020-05-18T21:40:32.564049+0000","last_clean":"2020-05-18T21:40:32.564049+0000","last_became_active":"2020-05-18T21:40:28.872462+0000","last_became_peered":"2020-05-18T21:40:28.872462+0000","last_unstale":"2020-05-18T21:40:32.564049+0000","last_undegraded":"2020-05-18T21:40:32.564049+0000","last_fullsized":"2020-05-18T21:40:32.564049+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'5","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:35.243812+0000","last_change":"2020-05-18T21:40:31.915725+0000","last_active":"2020-05-18T21:40:35.243812+0000","last_peered":"2020-05-18T21:40:35.243812+0000","last_clean":"2020-05-18T21:40:35.243812+0000","last_became_active":"2020-05-18T21:40:31.915194+0000","last_became_peered":"2020-05-18T21:40:31.915194+0000","last_unstale":"2020-05-18T21:40:35.243812+0000","last_undegraded":"2020-05-18T21:40:35.243812+0000","last_fullsized":"2020-05-18T21:40:35.243812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.572081+0000","last_change":"2020-05-18T21:40:32.571381+0000","last_active":"2020-05-18T21:40:32.572081+0000","last_peered":"2020-05-18T21:40:32.572081+0000","last_clean":"2020-05-18T21:40:32.572081+0000","last_became_active":"2020-05-18T21:40:32.570240+0000","last_became_peered":"2020-05-18T21:40:32.570240+0000","last_unstale":"2020-05-18T21:40:32.572081+0000","last_undegraded":"2020-05-18T21:40:32.572081+0000","last_fullsized":"2020-05-18T21:40:32.572081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'6","reported_seq":"16","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:35.251824+0000","last_change":"2020-05-18T21:40:31.915813+0000","last_active":"2020-05-18T21:40:35.251824+0000","last_peered":"2020-05-18T21:40:35.251824+0000","last_clean":"2020-05-18T21:40:35.251824+0000","last_became_active":"2020-05-18T21:40:31.915364+0000","last_became_peered":"2020-05-18T21:40:31.915364+0000","last_unstale":"2020-05-18T21:40:35.251824+0000","last_undegraded":"2020-05-18T21:40:35.251824+0000","last_fullsized":"2020-05-18T21:40:35.251824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.226918+0000","last_change":"2020-05-18T21:40:29.146687+0000","last_active":"2020-05-18T21:40:32.226918+0000","last_peered":"2020-05-18T21:40:32.226918+0000","last_clean":"2020-05-18T21:40:32.226918+0000","last_became_active":"2020-05-18T21:40:29.145593+0000","last_became_peered":"2020-05-18T21:40:29.145593+0000","last_unstale":"2020-05-18T21:40:32.226918+0000","last_undegraded":"2020-05-18T21:40:32.226918+0000","last_fullsized":"2020-05-18T21:40:32.226918+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.572300+0000","last_change":"2020-05-18T21:40:32.571628+0000","last_active":"2020-05-18T21:40:32.572300+0000","last_peered":"2020-05-18T21:40:32.572300+0000","last_clean":"2020-05-18T21:40:32.572300+0000","last_became_active":"2020-05-18T21:40:32.570196+0000","last_became_peered":"2020-05-18T21:40:32.570196+0000","last_unstale":"2020-05-18T21:40:32.572300+0000","last_undegraded":"2020-05-18T21:40:32.572300+0000","last_fullsized":"2020-05-18T21:40:32.572300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:35.258540+0000","last_change":"2020-05-18T21:40:31.913805+0000","last_active":"2020-05-18T21:40:35.258540+0000","last_peered":"2020-05-18T21:40:35.258540+0000","last_clean":"2020-05-18T21:40:35.258540+0000","last_became_active":"2020-05-18T21:40:31.913316+0000","last_became_peered":"2020-05-18T21:40:31.913316+0000","last_unstale":"2020-05-18T21:40:35.258540+0000","last_undegraded":"2020-05-18T21:40:35.258540+0000","last_fullsized":"2020-05-18T21:40:35.258540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1526,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.263188+0000","last_change":"2020-05-18T21:40:29.143099+0000","last_active":"2020-05-18T21:40:32.263188+0000","last_peered":"2020-05-18T21:40:32.263188+0000","last_clean":"2020-05-18T21:40:32.263188+0000","last_became_active":"2020-05-18T21:40:29.142006+0000","last_became_peered":"2020-05-18T21:40:29.142006+0000","last_unstale":"2020-05-18T21:40:32.263188+0000","last_undegraded":"2020-05-18T21:40:32.263188+0000","last_fullsized":"2020-05-18T21:40:32.263188+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.270725+0000","last_change":"2020-05-18T21:40:32.270082+0000","last_active":"2020-05-18T21:40:32.270725+0000","last_peered":"2020-05-18T21:40:32.270725+0000","last_clean":"2020-05-18T21:40:32.270725+0000","last_became_active":"2020-05-18T21:40:32.269535+0000","last_became_peered":"2020-05-18T21:40:32.269535+0000","last_unstale":"2020-05-18T21:40:32.270725+0000","last_undegraded":"2020-05-18T21:40:32.270725+0000","last_fullsized":"2020-05-18T21:40:32.270725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"22'7","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:35.257897+0000","last_change":"2020-05-18T21:40:31.921093+0000","last_active":"2020-05-18T21:40:35.257897+0000","last_peered":"2020-05-18T21:40:35.257897+0000","last_clean":"2020-05-18T21:40:35.257897+0000","last_became_active":"2020-05-18T21:40:31.920592+0000","last_became_peered":"2020-05-18T21:40:31.920592+0000","last_unstale":"2020-05-18T21:40:35.257897+0000","last_undegraded":"2020-05-18T21:40:35.257897+0000","last_fullsized":"2020-05-18T21:40:35.257897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.226335+0000","last_change":"2020-05-18T21:40:28.876605+0000","last_active":"2020-05-18T21:40:32.226335+0000","last_peered":"2020-05-18T21:40:32.226335+0000","last_clean":"2020-05-18T21:40:32.226335+0000","last_became_active":"2020-05-18T21:40:28.876211+0000","last_became_peered":"2020-05-18T21:40:28.876211+0000","last_unstale":"2020-05-18T21:40:32.226335+0000","last_undegraded":"2020-05-18T21:40:32.226335+0000","last_fullsized":"2020-05-18T21:40:32.226335+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.568141+0000","last_change":"2020-05-18T21:40:32.567431+0000","last_active":"2020-05-18T21:40:32.568141+0000","last_peered":"2020-05-18T21:40:32.568141+0000","last_clean":"2020-05-18T21:40:32.568141+0000","last_became_active":"2020-05-18T21:40:32.566930+0000","last_became_peered":"2020-05-18T21:40:32.566930+0000","last_unstale":"2020-05-18T21:40:32.568141+0000","last_undegraded":"2020-05-18T21:40:32.568141+0000","last_fullsized":"2020-05-18T21:40:32.568141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"22'3","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:35.250452+0000","last_change":"2020-05-18T21:40:31.922177+0000","last_active":"2020-05-18T21:40:35.250452+0000","last_peered":"2020-05-18T21:40:35.250452+0000","last_clean":"2020-05-18T21:40:35.250452+0000","last_became_active":"2020-05-18T21:40:31.921714+0000","last_became_peered":"2020-05-18T21:40:31.921714+0000","last_unstale":"2020-05-18T21:40:35.250452+0000","last_undegraded":"2020-05-18T21:40:35.250452+0000","last_fullsized":"2020-05-18T21:40:35.250452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.217627+0000","last_change":"2020-05-18T21:40:28.873647+0000","last_active":"2020-05-18T21:40:32.217627+0000","last_peered":"2020-05-18T21:40:32.217627+0000","last_clean":"2020-05-18T21:40:32.217627+0000","last_became_active":"2020-05-18T21:40:28.873201+0000","last_became_peered":"2020-05-18T21:40:28.873201+0000","last_unstale":"2020-05-18T21:40:32.217627+0000","last_undegraded":"2020-05-18T21:40:32.217627+0000","last_fullsized":"2020-05-18T21:40:32.217627+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":86559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3809999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.53}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.327}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0339999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9359999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048808,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323032,"statfs":{"total":96636764160,"available":95562784768,"internally_reserved":1073741824,"allocated":229376,"data_stored":85993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6839999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4470000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1589999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2320000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.46}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":86153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.048}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.266}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6439999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9510000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4129999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0110000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7309999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323028,"statfs":{"total":96636764160,"available":95562780672,"internally_reserved":1073741824,"allocated":233472,"data_stored":86027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5310000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2530000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2120000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.165}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7389999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048824,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323016,"statfs":{"total":96636764160,"available":95562768384,"internally_reserved":1073741824,"allocated":245760,"data_stored":86719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.851}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.206}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7769999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.423}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.544}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048820,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323020,"statfs":{"total":96636764160,"available":95562772480,"internally_reserved":1073741824,"allocated":241664,"data_stored":87519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.262}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5249999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6230000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1709999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.044}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048820,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323020,"statfs":{"total":96636764160,"available":95562772480,"internally_reserved":1073741824,"allocated":241664,"data_stored":87519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.04}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1030000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323028,"statfs":{"total":96636764160,"available":95562780672,"internally_reserved":1073741824,"allocated":233472,"data_stored":86027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7690000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.601}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5550000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9489999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0760000000000001}]}]}]}} 2020-05-18T21:40:41.467 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-05-18T21:40:41.638 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafb4b0d700 1 Processor -- start 2020-05-18T21:40:41.639 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafb4b0d700 1 -- start start 2020-05-18T21:40:41.640 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafb4b0d700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fafb00a4080 0x7fafb0140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:41.641 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafb4b0d700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fafb0140590 0x7fafb0146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:41.641 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafb4b0d700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb009e0b0 con 0x7fafb00a7220 2020-05-18T21:40:41.641 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafb4b0d700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb00a3580 con 0x7fafb00a4080 2020-05-18T21:40:41.641 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafb4b0d700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00a20f0 con 0x7fafb0140590 2020-05-18T21:40:41.641 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaeffd700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fafb00a4080 0x7fafb0140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.642 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaeffd700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fafb00a4080 0x7fafb0140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34780/0 (socket says 172.21.15.196:34780) 2020-05-18T21:40:41.642 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaeffd700 1 -- 172.21.15.196:0/2444398543 learned_addr learned my addr 172.21.15.196:0/2444398543 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:41.642 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaffff700 1 --2- 172.21.15.196:0/2444398543 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fafb0140590 0x7fafb0146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.642 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaeffd700 1 -- 172.21.15.196:0/2444398543 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fafb0140590 msgr2=0x7fafb0146fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:41.642 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaeffd700 1 --2- 172.21.15.196:0/2444398543 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fafb0140590 0x7fafb0146fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:41.643 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaeffd700 1 -- 172.21.15.196:0/2444398543 >> v1:172.21.15.80:6789/0 conn(0x7fafb00a7220 legacy=0x7fafb00a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.643 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaeffd700 1 -- 172.21.15.196:0/2444398543 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb00a6a60 con 0x7fafb00a4080 2020-05-18T21:40:41.643 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaeffd700 1 --2- 172.21.15.196:0/2444398543 >> v2:172.21.15.196:3300/0 conn(0x7fafb00a4080 0x7fafb0140130 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fafa80032f0 tx=0x7fafa8007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.643 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaeffd700 1 --2- 172.21.15.196:0/2444398543 >> v2:172.21.15.196:3300/0 conn(0x7fafb00a4080 0x7fafb0140130 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fafa80032f0 tx=0x7fafa8007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.643 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaeffd700 1 --2- 172.21.15.196:0/2444398543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb00a4080 0x7fafb0140130 secure :-1 s=READY pgs=170 cs=0 l=1 rx=0x7fafa80032f0 tx=0x7fafa8007e00).ready entity=mon.1 client_cookie=6c2ae117b595ce7d server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:41.644 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaeffd700 1 --2- 172.21.15.196:0/2444398543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb00a4080 0x7fafb0140130 secure :-1 s=THROTTLE_DONE pgs=170 cs=0 l=1 rx=0x7fafa80032f0 tx=0x7fafa8007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.644 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaeffd700 1 --2- 172.21.15.196:0/2444398543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb00a4080 0x7fafb0140130 secure :-1 s=THROTTLE_DONE pgs=170 cs=0 l=1 rx=0x7fafa80032f0 tx=0x7fafa8007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.644 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafaeffd700 1 --2- 172.21.15.196:0/2444398543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb00a4080 0x7fafb0140130 secure :-1 s=THROTTLE_DONE pgs=170 cs=0 l=1 rx=0x7fafa80032f0 tx=0x7fafa8007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.644 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafae7fc700 1 -- 172.21.15.196:0/2444398543 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafa8011d10 con 0x7fafb00a4080 2020-05-18T21:40:41.644 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafae7fc700 1 -- 172.21.15.196:0/2444398543 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa8011e70 con 0x7fafb00a4080 2020-05-18T21:40:41.645 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafae7fc700 1 -- 172.21.15.196:0/2444398543 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafa800a370 con 0x7fafb00a4080 2020-05-18T21:40:41.645 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafb4b0d700 1 -- 172.21.15.196:0/2444398543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb00a4080 msgr2=0x7fafb0140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.645 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.639+0000 7fafb4b0d700 1 --2- 172.21.15.196:0/2444398543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb00a4080 0x7fafb0140130 secure :-1 s=READY pgs=170 cs=0 l=1 rx=0x7fafa80032f0 tx=0x7fafa8007e00).stop 2020-05-18T21:40:41.648 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 -- 172.21.15.196:0/2444398543 shutdown_connections 2020-05-18T21:40:41.649 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 --2- 172.21.15.196:0/2444398543 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fafb0140590 0x7fafb0146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:41.649 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 --2- 172.21.15.196:0/2444398543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb00a4080 0x7fafb0140130 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.649 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 -- 172.21.15.196:0/2444398543 >> 172.21.15.196:0/2444398543 conn(0x7fafb005a7c0 msgr2=0x7fafb005abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:41.649 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 -- 172.21.15.196:0/2444398543 shutdown_connections 2020-05-18T21:40:41.649 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 -- 172.21.15.196:0/2444398543 wait complete. 2020-05-18T21:40:41.650 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 Processor -- start 2020-05-18T21:40:41.650 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 -- start start 2020-05-18T21:40:41.650 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fafb00a7220 0x7fafb0098410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:41.650 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fafb0140590 0x7fafb009a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:41.650 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb0097eb0 con 0x7fafb00a4080 2020-05-18T21:40:41.651 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb0098de0 con 0x7fafb00a7220 2020-05-18T21:40:41.651 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb0097970 con 0x7fafb0140590 2020-05-18T21:40:41.651 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaffff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fafb0140590 0x7fafb009a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.651 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaffff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fafb0140590 0x7fafb009a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34790/0 (socket says 172.21.15.196:34790) 2020-05-18T21:40:41.651 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaf7fe700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fafb00a4080 0x7fafb009d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43546/0 (socket says 172.21.15.196:43546) 2020-05-18T21:40:41.652 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaffff700 1 -- 172.21.15.196:0/341602096 learned_addr learned my addr 172.21.15.196:0/341602096 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:41.652 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaeffd700 1 --2- 172.21.15.196:0/341602096 >> v2:172.21.15.80:3300/0 conn(0x7fafb00a7220 0x7fafb0098410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.652 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafacff9700 1 -- 172.21.15.196:0/341602096 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858677531 0 0) 0x7fafb0097eb0 con 0x7fafb00a4080 2020-05-18T21:40:41.652 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaffff700 1 -- 172.21.15.196:0/341602096 >> v2:172.21.15.80:3300/0 conn(0x7fafb00a7220 msgr2=0x7fafb0098410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.652 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaffff700 1 --2- 172.21.15.196:0/341602096 >> v2:172.21.15.80:3300/0 conn(0x7fafb00a7220 0x7fafb0098410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.652 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaffff700 1 -- 172.21.15.196:0/341602096 >> v1:172.21.15.196:6789/0 conn(0x7fafb00a4080 legacy=0x7fafb009d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.653 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaffff700 1 -- 172.21.15.196:0/341602096 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb0097c10 con 0x7fafb0140590 2020-05-18T21:40:41.653 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaeffd700 1 --2- 172.21.15.196:0/341602096 >> v2:172.21.15.80:3300/0 conn(0x7fafb00a7220 0x7fafb0098410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:41.653 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaffff700 1 --2- 172.21.15.196:0/341602096 >> v2:172.21.15.196:3300/0 conn(0x7fafb0140590 0x7fafb009a010 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fafa0002f70 tx=0x7fafa0010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.653 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaffff700 1 --2- 172.21.15.196:0/341602096 >> v2:172.21.15.196:3300/0 conn(0x7fafb0140590 0x7fafb009a010 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fafa0002f70 tx=0x7fafa0010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.653 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaffff700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb0140590 0x7fafb009a010 secure :-1 s=READY pgs=172 cs=0 l=1 rx=0x7fafa0002f70 tx=0x7fafa0010c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:41.654 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaffff700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb0140590 0x7fafb009a010 secure :-1 s=THROTTLE_DONE pgs=172 cs=0 l=1 rx=0x7fafa0002f70 tx=0x7fafa0010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.654 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafacff9700 1 -- 172.21.15.196:0/341602096 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafa000eb50 con 0x7fafb0140590 2020-05-18T21:40:41.654 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaffff700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb0140590 0x7fafb009a010 secure :-1 s=THROTTLE_DONE pgs=172 cs=0 l=1 rx=0x7fafa0002f70 tx=0x7fafa0010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.654 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafaffff700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb0140590 0x7fafb009a010 secure :-1 s=THROTTLE_DONE pgs=172 cs=0 l=1 rx=0x7fafa0002f70 tx=0x7fafa0010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.654 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafacff9700 1 -- 172.21.15.196:0/341602096 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa000ecb0 con 0x7fafb0140590 2020-05-18T21:40:41.655 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 -- 172.21.15.196:0/341602096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb009c710 con 0x7fafb0140590 2020-05-18T21:40:41.655 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafacff9700 1 -- 172.21.15.196:0/341602096 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafa000eea0 con 0x7fafb0140590 2020-05-18T21:40:41.655 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.643+0000 7fafb4b0d700 1 -- 172.21.15.196:0/341602096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb01496c0 con 0x7fafb0140590 2020-05-18T21:40:41.655 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.647+0000 7fafaffff700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb0140590 0x7fafb009a010 secure :-1 s=THROTTLE_DONE pgs=172 cs=0 l=1 rx=0x7fafa0002f70 tx=0x7fafa0010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.655 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.647+0000 7fafb4b0d700 1 -- 172.21.15.196:0/341602096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb0097790 con 0x7fafb0140590 2020-05-18T21:40:41.655 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.647+0000 7fafacff9700 1 -- 172.21.15.196:0/341602096 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fafa0007190 con 0x7fafb0140590 2020-05-18T21:40:41.656 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.647+0000 7fafaffff700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb0140590 0x7fafb009a010 secure :-1 s=THROTTLE_DONE pgs=172 cs=0 l=1 rx=0x7fafa0002f70 tx=0x7fafa0010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.656 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.647+0000 7fafacff9700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7faf9404f930 0x7faf94051df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:41.656 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.647+0000 7fafacff9700 1 -- 172.21.15.196:0/341602096 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fafa0064160 con 0x7fafb0140590 2020-05-18T21:40:41.656 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.647+0000 7fafaf7fe700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7faf9404f930 0x7faf94051df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.656 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.647+0000 7fafaf7fe700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7faf9404f930 0x7faf94051df0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7faf980011e0 tx=0x7faf98001760).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.657 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.647+0000 7fafaf7fe700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7faf9404f930 0x7faf94051df0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7faf980011e0 tx=0x7faf98001760).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.657 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.647+0000 7fafaf7fe700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7faf9404f930 0x7faf94051df0 secure :-1 s=READY pgs=69 cs=0 l=1 rx=0x7faf980011e0 tx=0x7faf98001760).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:41.657 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.655+0000 7fafaffff700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb0140590 0x7fafb009a010 secure :-1 s=THROTTLE_DONE pgs=172 cs=0 l=1 rx=0x7fafa0002f70 tx=0x7fafa0010c30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.658 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.655+0000 7fafacff9700 1 -- 172.21.15.196:0/341602096 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7fafa003e4f0 con 0x7fafb0140590 2020-05-18T21:40:41.767 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.767+0000 7fafb4b0d700 1 -- 172.21.15.196:0/341602096 --> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fafb009d290 con 0x7faf9404f930 2020-05-18T21:40:41.775 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.775+0000 7fafaf7fe700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7faf9404f930 0x7faf94051df0 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7faf980011e0 tx=0x7faf98001760).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.776 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.775+0000 7fafacff9700 1 -- 172.21.15.196:0/341602096 <== mgr.4105 v2:172.21.15.196:6832/12657 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82205 (secure 0 0 0) 0x7fafb009d290 con 0x7faf9404f930 2020-05-18T21:40:41.776 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:41.777 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.775+0000 7fafb4b0d700 1 -- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7faf9404f930 msgr2=0x7faf94051df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.778 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.775+0000 7fafb4b0d700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7faf9404f930 0x7faf94051df0 secure :-1 s=READY pgs=69 cs=0 l=1 rx=0x7faf980011e0 tx=0x7faf98001760).stop 2020-05-18T21:40:41.778 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.775+0000 7fafb4b0d700 1 -- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb0140590 msgr2=0x7fafb009a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.778 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.775+0000 7fafb4b0d700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb0140590 0x7fafb009a010 secure :-1 s=READY pgs=172 cs=0 l=1 rx=0x7fafa0002f70 tx=0x7fafa0010c30).stop 2020-05-18T21:40:41.778 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.775+0000 7fafb4b0d700 1 -- 172.21.15.196:0/341602096 shutdown_connections 2020-05-18T21:40:41.778 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.775+0000 7fafb4b0d700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7faf9404f930 0x7faf94051df0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.779 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.775+0000 7fafb4b0d700 1 --2- 172.21.15.196:0/341602096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb0140590 0x7fafb009a010 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.779 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.775+0000 7fafb4b0d700 1 --2- 172.21.15.196:0/341602096 >> v2:172.21.15.80:3300/0 conn(0x7fafb00a7220 0x7fafb0098410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.779 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.775+0000 7fafb4b0d700 1 -- 172.21.15.196:0/341602096 >> 172.21.15.196:0/341602096 conn(0x7fafb005a7c0 msgr2=0x7fafb00a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:41.779 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.775+0000 7fafb4b0d700 1 -- 172.21.15.196:0/341602096 shutdown_connections 2020-05-18T21:40:41.779 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.775+0000 7fafb4b0d700 1 -- 172.21.15.196:0/341602096 wait complete. 2020-05-18T21:40:41.780 INFO:teuthology.orchestra.run.smithi196.stderr:dumped all 2020-05-18T21:40:41.789 INFO:teuthology.orchestra.run.smithi196.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2020-05-18T21:40:41.756202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390528,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584192,"statfs":{"total":773094113280,"available":764502212608,"internally_reserved":8589934592,"allocated":1900544,"data_stored":692516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.539240"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.222712+0000","last_change":"2020-05-18T21:40:32.221870+0000","last_active":"2020-05-18T21:40:32.222712+0000","last_peered":"2020-05-18T21:40:32.222712+0000","last_clean":"2020-05-18T21:40:32.222712+0000","last_became_active":"2020-05-18T21:40:32.221106+0000","last_became_peered":"2020-05-18T21:40:32.221106+0000","last_unstale":"2020-05-18T21:40:32.222712+0000","last_undegraded":"2020-05-18T21:40:32.222712+0000","last_fullsized":"2020-05-18T21:40:32.222712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"17","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.264464+0000","last_change":"2020-05-18T21:40:23.820420+0000","last_active":"2020-05-18T21:40:32.264464+0000","last_peered":"2020-05-18T21:40:32.264464+0000","last_clean":"2020-05-18T21:40:32.264464+0000","last_became_active":"2020-05-18T21:40:23.818790+0000","last_became_peered":"2020-05-18T21:40:23.818790+0000","last_unstale":"2020-05-18T21:40:32.264464+0000","last_undegraded":"2020-05-18T21:40:32.264464+0000","last_fullsized":"2020-05-18T21:40:32.264464+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:20.742680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:20.742680+0000","last_clean_scrub_stamp":"2020-05-18T21:40:20.742680+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.564050+0000","last_change":"2020-05-18T21:40:28.872619+0000","last_active":"2020-05-18T21:40:32.564050+0000","last_peered":"2020-05-18T21:40:32.564050+0000","last_clean":"2020-05-18T21:40:32.564050+0000","last_became_active":"2020-05-18T21:40:28.872176+0000","last_became_peered":"2020-05-18T21:40:28.872176+0000","last_unstale":"2020-05-18T21:40:32.564050+0000","last_undegraded":"2020-05-18T21:40:32.564050+0000","last_fullsized":"2020-05-18T21:40:32.564050+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"22'1","reported_seq":"11","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:33.541062+0000","last_change":"2020-05-18T21:40:31.921020+0000","last_active":"2020-05-18T21:40:33.541062+0000","last_peered":"2020-05-18T21:40:33.541062+0000","last_clean":"2020-05-18T21:40:33.541062+0000","last_became_active":"2020-05-18T21:40:31.920583+0000","last_became_peered":"2020-05-18T21:40:31.920583+0000","last_unstale":"2020-05-18T21:40:33.541062+0000","last_undegraded":"2020-05-18T21:40:33.541062+0000","last_fullsized":"2020-05-18T21:40:33.541062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.230558+0000","last_change":"2020-05-18T21:40:32.230208+0000","last_active":"2020-05-18T21:40:32.230558+0000","last_peered":"2020-05-18T21:40:32.230558+0000","last_clean":"2020-05-18T21:40:32.230558+0000","last_became_active":"2020-05-18T21:40:32.229548+0000","last_became_peered":"2020-05-18T21:40:32.229548+0000","last_unstale":"2020-05-18T21:40:32.230558+0000","last_undegraded":"2020-05-18T21:40:32.230558+0000","last_fullsized":"2020-05-18T21:40:32.230558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.264116+0000","last_change":"2020-05-18T21:40:28.870436+0000","last_active":"2020-05-18T21:40:32.264116+0000","last_peered":"2020-05-18T21:40:32.264116+0000","last_clean":"2020-05-18T21:40:32.264116+0000","last_became_active":"2020-05-18T21:40:28.870101+0000","last_became_peered":"2020-05-18T21:40:28.870101+0000","last_unstale":"2020-05-18T21:40:32.264116+0000","last_undegraded":"2020-05-18T21:40:32.264116+0000","last_fullsized":"2020-05-18T21:40:32.264116+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"10","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.226672+0000","last_change":"2020-05-18T21:40:31.919721+0000","last_active":"2020-05-18T21:40:32.226672+0000","last_peered":"2020-05-18T21:40:32.226672+0000","last_clean":"2020-05-18T21:40:32.226672+0000","last_became_active":"2020-05-18T21:40:31.918670+0000","last_became_peered":"2020-05-18T21:40:31.918670+0000","last_unstale":"2020-05-18T21:40:32.226672+0000","last_undegraded":"2020-05-18T21:40:32.226672+0000","last_fullsized":"2020-05-18T21:40:32.226672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.572101+0000","last_change":"2020-05-18T21:40:32.571510+0000","last_active":"2020-05-18T21:40:32.572101+0000","last_peered":"2020-05-18T21:40:32.572101+0000","last_clean":"2020-05-18T21:40:32.572101+0000","last_became_active":"2020-05-18T21:40:32.570722+0000","last_became_peered":"2020-05-18T21:40:32.570722+0000","last_unstale":"2020-05-18T21:40:32.572101+0000","last_undegraded":"2020-05-18T21:40:32.572101+0000","last_fullsized":"2020-05-18T21:40:32.572101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.219524+0000","last_change":"2020-05-18T21:40:28.873535+0000","last_active":"2020-05-18T21:40:32.219524+0000","last_peered":"2020-05-18T21:40:32.219524+0000","last_clean":"2020-05-18T21:40:32.219524+0000","last_became_active":"2020-05-18T21:40:28.872824+0000","last_became_peered":"2020-05-18T21:40:28.872824+0000","last_unstale":"2020-05-18T21:40:32.219524+0000","last_undegraded":"2020-05-18T21:40:32.219524+0000","last_fullsized":"2020-05-18T21:40:32.219524+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:35.257628+0000","last_change":"2020-05-18T21:40:31.915603+0000","last_active":"2020-05-18T21:40:35.257628+0000","last_peered":"2020-05-18T21:40:35.257628+0000","last_clean":"2020-05-18T21:40:35.257628+0000","last_became_active":"2020-05-18T21:40:31.913596+0000","last_became_peered":"2020-05-18T21:40:31.913596+0000","last_unstale":"2020-05-18T21:40:35.257628+0000","last_undegraded":"2020-05-18T21:40:35.257628+0000","last_fullsized":"2020-05-18T21:40:35.257628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.229937+0000","last_change":"2020-05-18T21:40:32.229478+0000","last_active":"2020-05-18T21:40:32.229937+0000","last_peered":"2020-05-18T21:40:32.229937+0000","last_clean":"2020-05-18T21:40:32.229937+0000","last_became_active":"2020-05-18T21:40:32.229111+0000","last_became_peered":"2020-05-18T21:40:32.229111+0000","last_unstale":"2020-05-18T21:40:32.229937+0000","last_undegraded":"2020-05-18T21:40:32.229937+0000","last_fullsized":"2020-05-18T21:40:32.229937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.564049+0000","last_change":"2020-05-18T21:40:28.872945+0000","last_active":"2020-05-18T21:40:32.564049+0000","last_peered":"2020-05-18T21:40:32.564049+0000","last_clean":"2020-05-18T21:40:32.564049+0000","last_became_active":"2020-05-18T21:40:28.872462+0000","last_became_peered":"2020-05-18T21:40:28.872462+0000","last_unstale":"2020-05-18T21:40:32.564049+0000","last_undegraded":"2020-05-18T21:40:32.564049+0000","last_fullsized":"2020-05-18T21:40:32.564049+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'5","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:35.243812+0000","last_change":"2020-05-18T21:40:31.915725+0000","last_active":"2020-05-18T21:40:35.243812+0000","last_peered":"2020-05-18T21:40:35.243812+0000","last_clean":"2020-05-18T21:40:35.243812+0000","last_became_active":"2020-05-18T21:40:31.915194+0000","last_became_peered":"2020-05-18T21:40:31.915194+0000","last_unstale":"2020-05-18T21:40:35.243812+0000","last_undegraded":"2020-05-18T21:40:35.243812+0000","last_fullsized":"2020-05-18T21:40:35.243812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.572081+0000","last_change":"2020-05-18T21:40:32.571381+0000","last_active":"2020-05-18T21:40:32.572081+0000","last_peered":"2020-05-18T21:40:32.572081+0000","last_clean":"2020-05-18T21:40:32.572081+0000","last_became_active":"2020-05-18T21:40:32.570240+0000","last_became_peered":"2020-05-18T21:40:32.570240+0000","last_unstale":"2020-05-18T21:40:32.572081+0000","last_undegraded":"2020-05-18T21:40:32.572081+0000","last_fullsized":"2020-05-18T21:40:32.572081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'6","reported_seq":"16","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:35.251824+0000","last_change":"2020-05-18T21:40:31.915813+0000","last_active":"2020-05-18T21:40:35.251824+0000","last_peered":"2020-05-18T21:40:35.251824+0000","last_clean":"2020-05-18T21:40:35.251824+0000","last_became_active":"2020-05-18T21:40:31.915364+0000","last_became_peered":"2020-05-18T21:40:31.915364+0000","last_unstale":"2020-05-18T21:40:35.251824+0000","last_undegraded":"2020-05-18T21:40:35.251824+0000","last_fullsized":"2020-05-18T21:40:35.251824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.226918+0000","last_change":"2020-05-18T21:40:29.146687+0000","last_active":"2020-05-18T21:40:32.226918+0000","last_peered":"2020-05-18T21:40:32.226918+0000","last_clean":"2020-05-18T21:40:32.226918+0000","last_became_active":"2020-05-18T21:40:29.145593+0000","last_became_peered":"2020-05-18T21:40:29.145593+0000","last_unstale":"2020-05-18T21:40:32.226918+0000","last_undegraded":"2020-05-18T21:40:32.226918+0000","last_fullsized":"2020-05-18T21:40:32.226918+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.572300+0000","last_change":"2020-05-18T21:40:32.571628+0000","last_active":"2020-05-18T21:40:32.572300+0000","last_peered":"2020-05-18T21:40:32.572300+0000","last_clean":"2020-05-18T21:40:32.572300+0000","last_became_active":"2020-05-18T21:40:32.570196+0000","last_became_peered":"2020-05-18T21:40:32.570196+0000","last_unstale":"2020-05-18T21:40:32.572300+0000","last_undegraded":"2020-05-18T21:40:32.572300+0000","last_fullsized":"2020-05-18T21:40:32.572300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:35.258540+0000","last_change":"2020-05-18T21:40:31.913805+0000","last_active":"2020-05-18T21:40:35.258540+0000","last_peered":"2020-05-18T21:40:35.258540+0000","last_clean":"2020-05-18T21:40:35.258540+0000","last_became_active":"2020-05-18T21:40:31.913316+0000","last_became_peered":"2020-05-18T21:40:31.913316+0000","last_unstale":"2020-05-18T21:40:35.258540+0000","last_undegraded":"2020-05-18T21:40:35.258540+0000","last_fullsized":"2020-05-18T21:40:35.258540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1526,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.263188+0000","last_change":"2020-05-18T21:40:29.143099+0000","last_active":"2020-05-18T21:40:32.263188+0000","last_peered":"2020-05-18T21:40:32.263188+0000","last_clean":"2020-05-18T21:40:32.263188+0000","last_became_active":"2020-05-18T21:40:29.142006+0000","last_became_peered":"2020-05-18T21:40:29.142006+0000","last_unstale":"2020-05-18T21:40:32.263188+0000","last_undegraded":"2020-05-18T21:40:32.263188+0000","last_fullsized":"2020-05-18T21:40:32.263188+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.270725+0000","last_change":"2020-05-18T21:40:32.270082+0000","last_active":"2020-05-18T21:40:32.270725+0000","last_peered":"2020-05-18T21:40:32.270725+0000","last_clean":"2020-05-18T21:40:32.270725+0000","last_became_active":"2020-05-18T21:40:32.269535+0000","last_became_peered":"2020-05-18T21:40:32.269535+0000","last_unstale":"2020-05-18T21:40:32.270725+0000","last_undegraded":"2020-05-18T21:40:32.270725+0000","last_fullsized":"2020-05-18T21:40:32.270725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"22'7","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:35.257897+0000","last_change":"2020-05-18T21:40:31.921093+0000","last_active":"2020-05-18T21:40:35.257897+0000","last_peered":"2020-05-18T21:40:35.257897+0000","last_clean":"2020-05-18T21:40:35.257897+0000","last_became_active":"2020-05-18T21:40:31.920592+0000","last_became_peered":"2020-05-18T21:40:31.920592+0000","last_unstale":"2020-05-18T21:40:35.257897+0000","last_undegraded":"2020-05-18T21:40:35.257897+0000","last_fullsized":"2020-05-18T21:40:35.257897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.226335+0000","last_change":"2020-05-18T21:40:28.876605+0000","last_active":"2020-05-18T21:40:32.226335+0000","last_peered":"2020-05-18T21:40:32.226335+0000","last_clean":"2020-05-18T21:40:32.226335+0000","last_became_active":"2020-05-18T21:40:28.876211+0000","last_became_peered":"2020-05-18T21:40:28.876211+0000","last_unstale":"2020-05-18T21:40:32.226335+0000","last_undegraded":"2020-05-18T21:40:32.226335+0000","last_fullsized":"2020-05-18T21:40:32.226335+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.568141+0000","last_change":"2020-05-18T21:40:32.567431+0000","last_active":"2020-05-18T21:40:32.568141+0000","last_peered":"2020-05-18T21:40:32.568141+0000","last_clean":"2020-05-18T21:40:32.568141+0000","last_became_active":"2020-05-18T21:40:32.566930+0000","last_became_peered":"2020-05-18T21:40:32.566930+0000","last_unstale":"2020-05-18T21:40:32.568141+0000","last_undegraded":"2020-05-18T21:40:32.568141+0000","last_fullsized":"2020-05-18T21:40:32.568141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:31.894231+0000","last_clean_scrub_stamp":"2020-05-18T21:40:31.894231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"22'3","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-05-18T21:40:35.250452+0000","last_change":"2020-05-18T21:40:31.922177+0000","last_active":"2020-05-18T21:40:35.250452+0000","last_peered":"2020-05-18T21:40:35.250452+0000","last_clean":"2020-05-18T21:40:35.250452+0000","last_became_active":"2020-05-18T21:40:31.921714+0000","last_became_peered":"2020-05-18T21:40:31.921714+0000","last_unstale":"2020-05-18T21:40:35.250452+0000","last_undegraded":"2020-05-18T21:40:35.250452+0000","last_fullsized":"2020-05-18T21:40:35.250452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:30.884191+0000","last_clean_scrub_stamp":"2020-05-18T21:40:30.884191+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2020-05-18T21:40:32.217627+0000","last_change":"2020-05-18T21:40:28.873647+0000","last_active":"2020-05-18T21:40:32.217627+0000","last_peered":"2020-05-18T21:40:32.217627+0000","last_clean":"2020-05-18T21:40:32.217627+0000","last_became_active":"2020-05-18T21:40:28.873201+0000","last_became_peered":"2020-05-18T21:40:28.873201+0000","last_unstale":"2020-05-18T21:40:32.217627+0000","last_undegraded":"2020-05-18T21:40:32.217627+0000","last_fullsized":"2020-05-18T21:40:32.217627+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-18T21:40:27.844562+0000","last_clean_scrub_stamp":"2020-05-18T21:40:27.844562+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":86559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3809999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.53}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.327}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0339999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9359999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048808,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323032,"statfs":{"total":96636764160,"available":95562784768,"internally_reserved":1073741824,"allocated":229376,"data_stored":85993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6839999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4470000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1589999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2320000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.46}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":86153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.048}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.266}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6439999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9510000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4129999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0110000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7309999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323028,"statfs":{"total":96636764160,"available":95562780672,"internally_reserved":1073741824,"allocated":233472,"data_stored":86027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5310000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2530000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2120000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.165}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7389999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048824,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323016,"statfs":{"total":96636764160,"available":95562768384,"internally_reserved":1073741824,"allocated":245760,"data_stored":86719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.851}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.206}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7769999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.423}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.544}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048820,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323020,"statfs":{"total":96636764160,"available":95562772480,"internally_reserved":1073741824,"allocated":241664,"data_stored":87519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.262}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5249999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6230000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1709999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.044}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048820,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323020,"statfs":{"total":96636764160,"available":95562772480,"internally_reserved":1073741824,"allocated":241664,"data_stored":87519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.04}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1030000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323028,"statfs":{"total":96636764160,"available":95562780672,"internally_reserved":1073741824,"allocated":233472,"data_stored":86027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7690000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.601}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5550000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9489999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0760000000000001}]}]}]}} 2020-05-18T21:40:41.792 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-05-18T21:40:41.792 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-05-18T21:40:41.793 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-05-18T21:40:41.793 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early health --format=json 2020-05-18T21:40:41.896 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.895+0000 7f7254a42700 1 Processor -- start 2020-05-18T21:40:41.896 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.895+0000 7f7254a42700 1 -- start start 2020-05-18T21:40:41.899 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.895+0000 7f7254a42700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f725013fd80 0x7f7250140210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:41.899 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.895+0000 7f7254a42700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7250140670 0x7f72501471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:41.899 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.895+0000 7f7254a42700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72500a7750 con 0x7f725005b210 2020-05-18T21:40:41.900 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.895+0000 7f7254a42700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72500a4e10 con 0x7f725013fd80 2020-05-18T21:40:41.900 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.895+0000 7f7254a42700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72500a7980 con 0x7f7250140670 2020-05-18T21:40:41.900 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.895+0000 7f724ffff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7250140670 0x7f72501471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.900 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.895+0000 7f724ffff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7250140670 0x7f72501471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34798/0 (socket says 172.21.15.196:34798) 2020-05-18T21:40:41.900 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.895+0000 7f724ffff700 1 -- 172.21.15.196:0/2330888471 learned_addr learned my addr 172.21.15.196:0/2330888471 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:41.900 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.895+0000 7f724effd700 1 --2- 172.21.15.196:0/2330888471 >> v2:172.21.15.80:3300/0 conn(0x7f725013fd80 0x7f7250140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.901 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f724ffff700 1 -- 172.21.15.196:0/2330888471 >> v2:172.21.15.80:3300/0 conn(0x7f725013fd80 msgr2=0x7f7250140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:41.901 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f724ffff700 1 --2- 172.21.15.196:0/2330888471 >> v2:172.21.15.80:3300/0 conn(0x7f725013fd80 0x7f7250140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:41.901 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f724ffff700 1 -- 172.21.15.196:0/2330888471 >> v1:172.21.15.80:6789/0 conn(0x7f725005b210 legacy=0x7f72500a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:41.901 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f724ffff700 1 -- 172.21.15.196:0/2330888471 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72500a7c20 con 0x7f7250140670 2020-05-18T21:40:41.901 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f724ffff700 1 --2- 172.21.15.196:0/2330888471 >> v2:172.21.15.196:3300/0 conn(0x7f7250140670 0x7f72501471e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f72500a22b0 tx=0x7f7240007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.902 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f724ffff700 1 --2- 172.21.15.196:0/2330888471 >> v2:172.21.15.196:3300/0 conn(0x7f7250140670 0x7f72501471e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f72500a22b0 tx=0x7f7240007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.902 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f724ffff700 1 --2- 172.21.15.196:0/2330888471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7250140670 0x7f72501471e0 secure :-1 s=READY pgs=173 cs=0 l=1 rx=0x7f72500a22b0 tx=0x7f7240007a80).ready entity=mon.1 client_cookie=2e6f0232ba372d06 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:41.902 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f724ffff700 1 --2- 172.21.15.196:0/2330888471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7250140670 0x7f72501471e0 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7f72500a22b0 tx=0x7f7240007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.902 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f724e7fc700 1 -- 172.21.15.196:0/2330888471 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7240011720 con 0x7f7250140670 2020-05-18T21:40:41.902 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f724ffff700 1 --2- 172.21.15.196:0/2330888471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7250140670 0x7f72501471e0 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7f72500a22b0 tx=0x7f7240007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.903 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f724e7fc700 1 -- 172.21.15.196:0/2330888471 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240011880 con 0x7f7250140670 2020-05-18T21:40:41.903 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f724ffff700 1 --2- 172.21.15.196:0/2330888471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7250140670 0x7f72501471e0 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7f72500a22b0 tx=0x7f7240007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.903 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f724e7fc700 1 -- 172.21.15.196:0/2330888471 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7240011c60 con 0x7f7250140670 2020-05-18T21:40:41.903 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f7254a42700 1 -- 172.21.15.196:0/2330888471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7250140670 msgr2=0x7f72501471e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.903 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f7254a42700 1 --2- 172.21.15.196:0/2330888471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7250140670 0x7f72501471e0 secure :-1 s=READY pgs=173 cs=0 l=1 rx=0x7f72500a22b0 tx=0x7f7240007a80).stop 2020-05-18T21:40:41.904 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f7254a42700 1 -- 172.21.15.196:0/2330888471 shutdown_connections 2020-05-18T21:40:41.904 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f7254a42700 1 --2- 172.21.15.196:0/2330888471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7250140670 0x7f72501471e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.904 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f7254a42700 1 --2- 172.21.15.196:0/2330888471 >> v2:172.21.15.80:3300/0 conn(0x7f725013fd80 0x7f7250140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:41.904 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f7254a42700 1 -- 172.21.15.196:0/2330888471 >> 172.21.15.196:0/2330888471 conn(0x7f7250060640 msgr2=0x7f725005a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:41.904 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f7254a42700 1 -- 172.21.15.196:0/2330888471 shutdown_connections 2020-05-18T21:40:41.905 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f7254a42700 1 -- 172.21.15.196:0/2330888471 wait complete. 2020-05-18T21:40:41.908 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f7254a42700 1 Processor -- start 2020-05-18T21:40:41.908 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.899+0000 7f7254a42700 1 -- start start 2020-05-18T21:40:41.908 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f7254a42700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f725005b210 0x7f725013b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:41.909 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f7254a42700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f725013fd80 0x7f72500a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:41.909 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f7254a42700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7250140670 0x7f725013ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:41.909 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f7254a42700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725013f1f0 con 0x7f7250140670 2020-05-18T21:40:41.909 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f7254a42700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725013b630 con 0x7f725005b210 2020-05-18T21:40:41.909 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f7254a42700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725013da90 con 0x7f725013fd80 2020-05-18T21:40:41.910 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f725005b210 0x7f725013b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.910 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f725005b210 0x7f725013b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34800/0 (socket says 172.21.15.196:34800) 2020-05-18T21:40:41.910 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 -- 172.21.15.196:0/2606919936 learned_addr learned my addr 172.21.15.196:0/2606919936 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:41.910 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724effd700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f725013fd80 0x7f72500a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.910 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724ffff700 1 --2- 172.21.15.196:0/2606919936 >> v2:172.21.15.80:3300/0 conn(0x7f7250140670 0x7f725013ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.910 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 -- 172.21.15.196:0/2606919936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f725013fd80 msgr2=0x7f72500a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.911 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f725013fd80 0x7f72500a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.911 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 -- 172.21.15.196:0/2606919936 >> v2:172.21.15.80:3300/0 conn(0x7f7250140670 msgr2=0x7f725013ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:41.911 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 --2- 172.21.15.196:0/2606919936 >> v2:172.21.15.80:3300/0 conn(0x7f7250140670 0x7f725013ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:41.911 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 -- 172.21.15.196:0/2606919936 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725013dd30 con 0x7f725005b210 2020-05-18T21:40:41.911 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724ffff700 1 --2- 172.21.15.196:0/2606919936 >> v2:172.21.15.80:3300/0 conn(0x7f7250140670 0x7f725013ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request state changed! 2020-05-18T21:40:41.912 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 --2- 172.21.15.196:0/2606919936 >> v2:172.21.15.196:3300/0 conn(0x7f725005b210 0x7f725013b150 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7238001f90 tx=0x7f7238010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.912 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 --2- 172.21.15.196:0/2606919936 >> v2:172.21.15.196:3300/0 conn(0x7f725005b210 0x7f725013b150 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7238001f90 tx=0x7f7238010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.912 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725005b210 0x7f725013b150 secure :-1 s=READY pgs=174 cs=0 l=1 rx=0x7f7238001f90 tx=0x7f7238010a50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:41.912 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725005b210 0x7f725013b150 secure :-1 s=THROTTLE_DONE pgs=174 cs=0 l=1 rx=0x7f7238001f90 tx=0x7f7238010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.912 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725005b210 0x7f725013b150 secure :-1 s=THROTTLE_DONE pgs=174 cs=0 l=1 rx=0x7f7238001f90 tx=0x7f7238010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.913 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724f7fe700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725005b210 0x7f725013b150 secure :-1 s=THROTTLE_DONE pgs=174 cs=0 l=1 rx=0x7f7238001f90 tx=0x7f7238010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.913 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724cff9700 1 -- 172.21.15.196:0/2606919936 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f723800cc60 con 0x7f725005b210 2020-05-18T21:40:41.913 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724cff9700 1 -- 172.21.15.196:0/2606919936 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723800cdc0 con 0x7f725005b210 2020-05-18T21:40:41.913 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f7254a42700 1 -- 172.21.15.196:0/2606919936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725013d710 con 0x7f725005b210 2020-05-18T21:40:41.913 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f724cff9700 1 -- 172.21.15.196:0/2606919936 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f723800e370 con 0x7f725005b210 2020-05-18T21:40:41.914 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.903+0000 7f7254a42700 1 -- 172.21.15.196:0/2606919936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725013bec0 con 0x7f725005b210 2020-05-18T21:40:41.914 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.907+0000 7f724f7fe700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725005b210 0x7f725013b150 secure :-1 s=THROTTLE_DONE pgs=174 cs=0 l=1 rx=0x7f7238001f90 tx=0x7f7238010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.914 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.907+0000 7f7254a42700 1 -- 172.21.15.196:0/2606919936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725013b770 con 0x7f725005b210 2020-05-18T21:40:41.914 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.907+0000 7f724cff9700 1 -- 172.21.15.196:0/2606919936 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f7238006520 con 0x7f725005b210 2020-05-18T21:40:41.914 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.907+0000 7f724f7fe700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725005b210 0x7f725013b150 secure :-1 s=THROTTLE_DONE pgs=174 cs=0 l=1 rx=0x7f7238001f90 tx=0x7f7238010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.914 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.907+0000 7f724cff9700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f723404fa50 0x7f7234051f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:41.915 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.907+0000 7f724cff9700 1 -- 172.21.15.196:0/2606919936 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7238063b40 con 0x7f725005b210 2020-05-18T21:40:41.915 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.907+0000 7f724effd700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f723404fa50 0x7f7234051f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:41.915 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.907+0000 7f724effd700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f723404fa50 0x7f7234051f10 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f72480032f0 tx=0x7f7248007ac0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.915 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.907+0000 7f724effd700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f723404fa50 0x7f7234051f10 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f72480032f0 tx=0x7f7248007ac0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.915 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.907+0000 7f724effd700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f723404fa50 0x7f7234051f10 secure :-1 s=READY pgs=70 cs=0 l=1 rx=0x7f72480032f0 tx=0x7f7248007ac0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:41.917 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.915+0000 7f724f7fe700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725005b210 0x7f725013b150 secure :-1 s=THROTTLE_DONE pgs=174 cs=0 l=1 rx=0x7f7238001f90 tx=0x7f7238010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:41.918 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:41.915+0000 7f724cff9700 1 -- 172.21.15.196:0/2606919936 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f723803de70 con 0x7f725005b210 2020-05-18T21:40:42.065 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.063+0000 7f7254a42700 1 -- 172.21.15.196:0/2606919936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f72500569e0 con 0x7f725005b210 2020-05-18T21:40:42.066 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f724f7fe700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725005b210 0x7f725013b150 secure :-1 s=THROTTLE_DONE pgs=174 cs=0 l=1 rx=0x7f7238001f90 tx=0x7f7238010a50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.066 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f724cff9700 1 -- 172.21.15.196:0/2606919936 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f7238041490 con 0x7f725005b210 2020-05-18T21:40:42.066 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:42.067 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f7254a42700 1 -- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f723404fa50 msgr2=0x7f7234051f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f7254a42700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f723404fa50 0x7f7234051f10 secure :-1 s=READY pgs=70 cs=0 l=1 rx=0x7f72480032f0 tx=0x7f7248007ac0).stop 2020-05-18T21:40:42.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f7254a42700 1 -- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725005b210 msgr2=0x7f725013b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f7254a42700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725005b210 0x7f725013b150 secure :-1 s=READY pgs=174 cs=0 l=1 rx=0x7f7238001f90 tx=0x7f7238010a50).stop 2020-05-18T21:40:42.068 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f7254a42700 1 -- 172.21.15.196:0/2606919936 shutdown_connections 2020-05-18T21:40:42.069 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f7254a42700 1 --2- 172.21.15.196:0/2606919936 >> v2:172.21.15.80:3300/0 conn(0x7f7250140670 0x7f725013ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.069 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f7254a42700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f723404fa50 0x7f7234051f10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.069 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f7254a42700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f725013fd80 0x7f72500a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.069 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f7254a42700 1 --2- 172.21.15.196:0/2606919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725005b210 0x7f725013b150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.069 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f7254a42700 1 -- 172.21.15.196:0/2606919936 >> 172.21.15.196:0/2606919936 conn(0x7f7250060640 msgr2=0x7f72500a3280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:42.070 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f7254a42700 1 -- 172.21.15.196:0/2606919936 shutdown_connections 2020-05-18T21:40:42.070 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.067+0000 7f7254a42700 1 -- 172.21.15.196:0/2606919936 wait complete. 2020-05-18T21:40:42.078 INFO:teuthology.orchestra.run.smithi196.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-05-18T21:40:42.078 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-05-18T21:40:42.079 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-05-18T21:40:42.182 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 Processor -- start 2020-05-18T21:40:42.182 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 -- start start 2020-05-18T21:40:42.186 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f169005ab00 0x7f16900a6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:42.186 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f16900a1970 0x7f16900a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:42.186 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16900a3f10 con 0x7f169013feb0 2020-05-18T21:40:42.186 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169005b5c0 con 0x7f16900a1970 2020-05-18T21:40:42.187 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169005b860 con 0x7f169005ab00 2020-05-18T21:40:42.187 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1694de4700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f169005ab00 0x7f16900a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.187 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1694de4700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f169005ab00 0x7f16900a6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34808/0 (socket says 172.21.15.196:34808) 2020-05-18T21:40:42.187 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1694de4700 1 -- 172.21.15.196:0/63918087 learned_addr learned my addr 172.21.15.196:0/63918087 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:42.187 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f168ffff700 1 --2- 172.21.15.196:0/63918087 >> v2:172.21.15.80:3300/0 conn(0x7f16900a1970 0x7f16900a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.187 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1694de4700 1 -- 172.21.15.196:0/63918087 >> v2:172.21.15.80:3300/0 conn(0x7f16900a1970 msgr2=0x7f16900a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:42.188 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1694de4700 1 --2- 172.21.15.196:0/63918087 >> v2:172.21.15.80:3300/0 conn(0x7f16900a1970 0x7f16900a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:42.188 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1694de4700 1 -- 172.21.15.196:0/63918087 >> v1:172.21.15.80:6789/0 conn(0x7f169013feb0 legacy=0x7f16901402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.188 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1694de4700 1 -- 172.21.15.196:0/63918087 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16900a6060 con 0x7f169005ab00 2020-05-18T21:40:42.188 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1694de4700 1 --2- 172.21.15.196:0/63918087 >> v2:172.21.15.196:3300/0 conn(0x7f169005ab00 0x7f16900a6800 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.188 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1694de4700 1 --2- 172.21.15.196:0/63918087 >> v2:172.21.15.196:3300/0 conn(0x7f169005ab00 0x7f16900a6800 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.189 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1694de4700 1 --2- 172.21.15.196:0/63918087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f16900a6800 secure :-1 s=READY pgs=175 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).ready entity=mon.1 client_cookie=4ad38ee1e34646ae server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:42.189 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1694de4700 1 --2- 172.21.15.196:0/63918087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f16900a6800 secure :-1 s=THROTTLE_DONE pgs=175 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.189 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1694de4700 1 --2- 172.21.15.196:0/63918087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f16900a6800 secure :-1 s=THROTTLE_DONE pgs=175 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.189 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f168f7fe700 1 -- 172.21.15.196:0/63918087 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16800116b0 con 0x7f169005ab00 2020-05-18T21:40:42.189 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1694de4700 1 --2- 172.21.15.196:0/63918087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f16900a6800 secure :-1 s=THROTTLE_DONE pgs=175 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.190 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f168f7fe700 1 -- 172.21.15.196:0/63918087 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1680011810 con 0x7f169005ab00 2020-05-18T21:40:42.190 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f168f7fe700 1 -- 172.21.15.196:0/63918087 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1680011c30 con 0x7f169005ab00 2020-05-18T21:40:42.190 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 -- 172.21.15.196:0/63918087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 msgr2=0x7f16900a6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.190 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 --2- 172.21.15.196:0/63918087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f16900a6800 secure :-1 s=READY pgs=175 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).stop 2020-05-18T21:40:42.190 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 -- 172.21.15.196:0/63918087 shutdown_connections 2020-05-18T21:40:42.191 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 --2- 172.21.15.196:0/63918087 >> v2:172.21.15.80:3300/0 conn(0x7f16900a1970 0x7f16900a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:42.191 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 --2- 172.21.15.196:0/63918087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f16900a6800 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.191 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 -- 172.21.15.196:0/63918087 >> 172.21.15.196:0/63918087 conn(0x7f16900aa600 msgr2=0x7f16900aa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:42.191 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 -- 172.21.15.196:0/63918087 shutdown_connections 2020-05-18T21:40:42.191 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.183+0000 7f1697048700 1 -- 172.21.15.196:0/63918087 wait complete. 2020-05-18T21:40:42.194 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.187+0000 7f1697048700 1 Processor -- start 2020-05-18T21:40:42.195 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.187+0000 7f1697048700 1 -- start start 2020-05-18T21:40:42.195 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.187+0000 7f1697048700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f169005ab00 0x7f1690066de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:42.195 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.187+0000 7f1697048700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f169013feb0 0x7f1690098830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:42.195 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.187+0000 7f1697048700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16901499b0 con 0x7f16900a1970 2020-05-18T21:40:42.195 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.187+0000 7f1697048700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1690098db0 con 0x7f169013feb0 2020-05-18T21:40:42.196 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.187+0000 7f1697048700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1690099050 con 0x7f169005ab00 2020-05-18T21:40:42.196 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.187+0000 7f1694de4700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f169005ab00 0x7f1690066de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.196 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.187+0000 7f168ffff700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f16900a1970 0x7f169014a150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43576/0 (socket says 172.21.15.196:43576) 2020-05-18T21:40:42.196 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.187+0000 7f168ffff700 1 -- 172.21.15.196:0/2599560148 learned_addr learned my addr 172.21.15.196:0/2599560148 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:42.196 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.187+0000 7f16955e5700 1 --2- 172.21.15.196:0/2599560148 >> v2:172.21.15.80:3300/0 conn(0x7f169013feb0 0x7f1690098830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.197 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f168dffb700 1 -- 172.21.15.196:0/2599560148 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 781441166 0 0) 0x7f1688001250 con 0x7f16900a1970 2020-05-18T21:40:42.197 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1694de4700 1 -- 172.21.15.196:0/2599560148 >> v2:172.21.15.80:3300/0 conn(0x7f169013feb0 msgr2=0x7f1690098830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.197 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1694de4700 1 --2- 172.21.15.196:0/2599560148 >> v2:172.21.15.80:3300/0 conn(0x7f169013feb0 0x7f1690098830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.197 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1694de4700 1 -- 172.21.15.196:0/2599560148 >> v1:172.21.15.196:6789/0 conn(0x7f16900a1970 legacy=0x7f169014a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.197 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1694de4700 1 -- 172.21.15.196:0/2599560148 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16900992f0 con 0x7f169005ab00 2020-05-18T21:40:42.197 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1694de4700 1 --2- 172.21.15.196:0/2599560148 >> v2:172.21.15.196:3300/0 conn(0x7f169005ab00 0x7f1690066de0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.198 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1694de4700 1 --2- 172.21.15.196:0/2599560148 >> v2:172.21.15.196:3300/0 conn(0x7f169005ab00 0x7f1690066de0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.198 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1694de4700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f1690066de0 secure :-1 s=READY pgs=177 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:42.198 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1694de4700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f1690066de0 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.198 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f168dffb700 1 -- 172.21.15.196:0/2599560148 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1680011ea0 con 0x7f169005ab00 2020-05-18T21:40:42.198 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1694de4700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f1690066de0 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.199 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f168dffb700 1 -- 172.21.15.196:0/2599560148 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1680007360 con 0x7f169005ab00 2020-05-18T21:40:42.199 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1694de4700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f1690066de0 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.199 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1697048700 1 -- 172.21.15.196:0/2599560148 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16900a0c50 con 0x7f169005ab00 2020-05-18T21:40:42.199 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f168dffb700 1 -- 172.21.15.196:0/2599560148 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1680011cf0 con 0x7f169005ab00 2020-05-18T21:40:42.199 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1697048700 1 -- 172.21.15.196:0/2599560148 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169009a6b0 con 0x7f169005ab00 2020-05-18T21:40:42.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1694de4700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f1690066de0 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1697048700 1 -- 172.21.15.196:0/2599560148 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1690097710 con 0x7f169005ab00 2020-05-18T21:40:42.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f168dffb700 1 -- 172.21.15.196:0/2599560148 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f1680005700 con 0x7f169005ab00 2020-05-18T21:40:42.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f1694de4700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f1690066de0 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f168dffb700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f167404f860 0x7f1674051d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:42.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f168dffb700 1 -- 172.21.15.196:0/2599560148 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1680064220 con 0x7f169005ab00 2020-05-18T21:40:42.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f168ffff700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f167404f860 0x7f1674051d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f168ffff700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f167404f860 0x7f1674051d20 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1688008360 tx=0x7f16880015a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.191+0000 7f168ffff700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f167404f860 0x7f1674051d20 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1688008360 tx=0x7f16880015a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.195+0000 7f168ffff700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f167404f860 0x7f1674051d20 secure :-1 s=READY pgs=71 cs=0 l=1 rx=0x7f1688008360 tx=0x7f16880015a0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:42.202 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.199+0000 7f1694de4700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f1690066de0 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.202 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.199+0000 7f168dffb700 1 -- 172.21.15.196:0/2599560148 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f168003e610 con 0x7f169005ab00 2020-05-18T21:40:42.311 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.311+0000 7f1697048700 1 -- 172.21.15.196:0/2599560148 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16900569e0 con 0x7f169005ab00 2020-05-18T21:40:42.312 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.311+0000 7f1694de4700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f1690066de0 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.312 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.311+0000 7f168dffb700 1 -- 172.21.15.196:0/2599560148 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16089 (secure 0 0 0) 0x7f1680041c30 con 0x7f169005ab00 2020-05-18T21:40:42.313 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:42.313 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"fsid":"d048673a-3d91-4cc6-882d-e9cc70e7cd6c","created":"2020-05-18T21:40:15.911738+0000","modified":"2020-05-18T21:40:33.211283+0000","last_up_change":"2020-05-18T21:40:22.762927+0000","last_in_change":"2020-05-18T21:40:22.762927+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-05-18T21:40:19.808941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-05-18T21:40:26.944311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-05-18T21:40:30.135661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-05-18T21:40:31.183488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8db63a0a-61dc-42b5-9559-fddcf436cab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":12771},{"type":"v1","addr":"172.21.15.196:6801","nonce":12771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":12771},{"type":"v1","addr":"172.21.15.196:6803","nonce":12771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":12771},{"type":"v1","addr":"172.21.15.196:6807","nonce":12771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":12771},{"type":"v1","addr":"172.21.15.196:6805","nonce":12771}]},"public_addr":"172.21.15.196:6801/12771","cluster_addr":"172.21.15.196:6803/12771","heartbeat_back_addr":"172.21.15.196:6807/12771","heartbeat_front_addr":"172.21.15.196:6805/12771","state":["exists","up"]},{"osd":1,"uuid":"6773b0c3-9f49-42a2-b75b-c10c734ff086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":12772},{"type":"v1","addr":"172.21.15.196:6817","nonce":12772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":12772},{"type":"v1","addr":"172.21.15.196:6819","nonce":12772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":12772},{"type":"v1","addr":"172.21.15.196:6823","nonce":12772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":12772},{"type":"v1","addr":"172.21.15.196:6821","nonce":12772}]},"public_addr":"172.21.15.196:6817/12772","cluster_addr":"172.21.15.196:6819/12772","heartbeat_back_addr":"172.21.15.196:6823/12772","heartbeat_front_addr":"172.21.15.196:6821/12772","state":["exists","up"]},{"osd":2,"uuid":"ac1f2e05-9bb7-4768-b712-10e4d2ea4154","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":12773},{"type":"v1","addr":"172.21.15.196:6809","nonce":12773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":12773},{"type":"v1","addr":"172.21.15.196:6811","nonce":12773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":12773},{"type":"v1","addr":"172.21.15.196:6815","nonce":12773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":12773},{"type":"v1","addr":"172.21.15.196:6813","nonce":12773}]},"public_addr":"172.21.15.196:6809/12773","cluster_addr":"172.21.15.196:6811/12773","heartbeat_back_addr":"172.21.15.196:6815/12773","heartbeat_front_addr":"172.21.15.196:6813/12773","state":["exists","up"]},{"osd":3,"uuid":"bae5ab40-235b-4e0f-8d0d-524afd099b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":12775},{"type":"v1","addr":"172.21.15.196:6825","nonce":12775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":12775},{"type":"v1","addr":"172.21.15.196:6827","nonce":12775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":12775},{"type":"v1","addr":"172.21.15.196:6831","nonce":12775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":12775},{"type":"v1","addr":"172.21.15.196:6829","nonce":12775}]},"public_addr":"172.21.15.196:6825/12775","cluster_addr":"172.21.15.196:6827/12775","heartbeat_back_addr":"172.21.15.196:6831/12775","heartbeat_front_addr":"172.21.15.196:6829/12775","state":["exists","up"]},{"osd":4,"uuid":"275fc52c-58e4-4f35-a0cb-e117fe36bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":13040},{"type":"v1","addr":"172.21.15.80:6819","nonce":13040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6821","nonce":13040},{"type":"v1","addr":"172.21.15.80:6823","nonce":13040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6829","nonce":13040},{"type":"v1","addr":"172.21.15.80:6831","nonce":13040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":13040},{"type":"v1","addr":"172.21.15.80:6827","nonce":13040}]},"public_addr":"172.21.15.80:6819/13040","cluster_addr":"172.21.15.80:6823/13040","heartbeat_back_addr":"172.21.15.80:6831/13040","heartbeat_front_addr":"172.21.15.80:6827/13040","state":["exists","up"]},{"osd":5,"uuid":"9795b89c-e2f8-4abf-b9c7-796727a57002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":13037},{"type":"v1","addr":"172.21.15.80:6818","nonce":13037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":13037},{"type":"v1","addr":"172.21.15.80:6822","nonce":13037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":13037},{"type":"v1","addr":"172.21.15.80:6830","nonce":13037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":13037},{"type":"v1","addr":"172.21.15.80:6826","nonce":13037}]},"public_addr":"172.21.15.80:6818/13037","cluster_addr":"172.21.15.80:6822/13037","heartbeat_back_addr":"172.21.15.80:6830/13037","heartbeat_front_addr":"172.21.15.80:6826/13037","state":["exists","up"]},{"osd":6,"uuid":"e42df2e8-9cc5-4c7e-a243-bd18ccfbd002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":13039},{"type":"v1","addr":"172.21.15.80:6804","nonce":13039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":13039},{"type":"v1","addr":"172.21.15.80:6808","nonce":13039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":13039},{"type":"v1","addr":"172.21.15.80:6813","nonce":13039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":13039},{"type":"v1","addr":"172.21.15.80:6810","nonce":13039}]},"public_addr":"172.21.15.80:6804/13039","cluster_addr":"172.21.15.80:6808/13039","heartbeat_back_addr":"172.21.15.80:6813/13039","heartbeat_front_addr":"172.21.15.80:6810/13039","state":["exists","up"]},{"osd":7,"uuid":"ebd6c938-beff-4668-b70e-076ed22bb8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":13038},{"type":"v1","addr":"172.21.15.80:6801","nonce":13038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":13038},{"type":"v1","addr":"172.21.15.80:6805","nonce":13038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":13038},{"type":"v1","addr":"172.21.15.80:6817","nonce":13038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":13038},{"type":"v1","addr":"172.21.15.80:6811","nonce":13038}]},"public_addr":"172.21.15.80:6801/13038","cluster_addr":"172.21.15.80:6805/13038","heartbeat_back_addr":"172.21.15.80:6817/13038","heartbeat_front_addr":"172.21.15.80:6811/13038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.134515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.439836+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.476825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.176642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.452301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.427183+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.230250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.210168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-18T21:40:42.315 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.315+0000 7f1697048700 1 -- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f167404f860 msgr2=0x7f1674051d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.315 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.315+0000 7f1697048700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f167404f860 0x7f1674051d20 secure :-1 s=READY pgs=71 cs=0 l=1 rx=0x7f1688008360 tx=0x7f16880015a0).stop 2020-05-18T21:40:42.315 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.315+0000 7f1697048700 1 -- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 msgr2=0x7f1690066de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.315 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.315+0000 7f1697048700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f1690066de0 secure :-1 s=READY pgs=177 cs=0 l=1 rx=0x7f1680002f70 tx=0x7f1680007a80).stop 2020-05-18T21:40:42.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.315+0000 7f1697048700 1 -- 172.21.15.196:0/2599560148 shutdown_connections 2020-05-18T21:40:42.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.315+0000 7f1697048700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f167404f860 0x7f1674051d20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.315+0000 7f1697048700 1 --2- 172.21.15.196:0/2599560148 >> v2:172.21.15.80:3300/0 conn(0x7f169013feb0 0x7f1690098830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.315+0000 7f1697048700 1 --2- 172.21.15.196:0/2599560148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f169005ab00 0x7f1690066de0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.316 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.315+0000 7f1697048700 1 -- 172.21.15.196:0/2599560148 >> 172.21.15.196:0/2599560148 conn(0x7f16900aa600 msgr2=0x7f169005eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:42.317 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.315+0000 7f1697048700 1 -- 172.21.15.196:0/2599560148 shutdown_connections 2020-05-18T21:40:42.317 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.315+0000 7f1697048700 1 -- 172.21.15.196:0/2599560148 wait complete. 2020-05-18T21:40:42.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-05-18T21:40:19.808941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-05-18T21:40:26.944311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-05-18T21:40:30.135661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-05-18T21:40:31.183488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-05-18T21:40:42.331 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-05-18T21:40:42.434 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f2670bc8700 1 Processor -- start 2020-05-18T21:40:42.434 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f2670bc8700 1 -- start start 2020-05-18T21:40:42.435 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f2670bc8700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f266c0aae20 0x7f266c05c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:42.435 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f2670bc8700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f266c13bb70 0x7f266c13c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:42.436 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f2670bc8700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266c05c230 con 0x7f266c0af080 2020-05-18T21:40:42.436 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f2670bc8700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266c0a8760 con 0x7f266c13bb70 2020-05-18T21:40:42.436 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f2670bc8700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266c0a8a00 con 0x7f266c0aae20 2020-05-18T21:40:42.436 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266affd700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f266c0aae20 0x7f266c05c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.436 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266affd700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f266c0aae20 0x7f266c05c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34824/0 (socket says 172.21.15.196:34824) 2020-05-18T21:40:42.437 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266affd700 1 -- 172.21.15.196:0/2615531915 learned_addr learned my addr 172.21.15.196:0/2615531915 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:42.437 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266bfff700 1 --2- 172.21.15.196:0/2615531915 >> v2:172.21.15.80:3300/0 conn(0x7f266c13bb70 0x7f266c13c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.439 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266affd700 1 -- 172.21.15.196:0/2615531915 >> v2:172.21.15.80:3300/0 conn(0x7f266c13bb70 msgr2=0x7f266c13c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:42.440 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266affd700 1 --2- 172.21.15.196:0/2615531915 >> v2:172.21.15.80:3300/0 conn(0x7f266c13bb70 0x7f266c13c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:42.440 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266affd700 1 -- 172.21.15.196:0/2615531915 >> v1:172.21.15.80:6789/0 conn(0x7f266c0af080 legacy=0x7f266c0aa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.440 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266affd700 1 -- 172.21.15.196:0/2615531915 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c0a8ca0 con 0x7f266c0aae20 2020-05-18T21:40:42.440 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266affd700 1 --2- 172.21.15.196:0/2615531915 >> v2:172.21.15.196:3300/0 conn(0x7f266c0aae20 0x7f266c05c660 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f2654002f70 tx=0x7f2654007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266affd700 1 --2- 172.21.15.196:0/2615531915 >> v2:172.21.15.196:3300/0 conn(0x7f266c0aae20 0x7f266c05c660 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f2654002f70 tx=0x7f2654007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266affd700 1 --2- 172.21.15.196:0/2615531915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f266c0aae20 0x7f266c05c660 secure :-1 s=READY pgs=178 cs=0 l=1 rx=0x7f2654002f70 tx=0x7f2654007a80).ready entity=mon.1 client_cookie=b337364c0ec15afd server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:42.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266affd700 1 --2- 172.21.15.196:0/2615531915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f266c0aae20 0x7f266c05c660 secure :-1 s=THROTTLE_DONE pgs=178 cs=0 l=1 rx=0x7f2654002f70 tx=0x7f2654007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266affd700 1 --2- 172.21.15.196:0/2615531915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f266c0aae20 0x7f266c05c660 secure :-1 s=THROTTLE_DONE pgs=178 cs=0 l=1 rx=0x7f2654002f70 tx=0x7f2654007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266a7fc700 1 -- 172.21.15.196:0/2615531915 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26540116b0 con 0x7f266c0aae20 2020-05-18T21:40:42.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266affd700 1 --2- 172.21.15.196:0/2615531915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f266c0aae20 0x7f266c05c660 secure :-1 s=THROTTLE_DONE pgs=178 cs=0 l=1 rx=0x7f2654002f70 tx=0x7f2654007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266a7fc700 1 -- 172.21.15.196:0/2615531915 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2654011810 con 0x7f266c0aae20 2020-05-18T21:40:42.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f266a7fc700 1 -- 172.21.15.196:0/2615531915 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2654011c30 con 0x7f266c0aae20 2020-05-18T21:40:42.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f2670bc8700 1 -- 172.21.15.196:0/2615531915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f266c0aae20 msgr2=0x7f266c05c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.435+0000 7f2670bc8700 1 --2- 172.21.15.196:0/2615531915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f266c0aae20 0x7f266c05c660 secure :-1 s=READY pgs=178 cs=0 l=1 rx=0x7f2654002f70 tx=0x7f2654007a80).stop 2020-05-18T21:40:42.442 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2670bc8700 1 -- 172.21.15.196:0/2615531915 shutdown_connections 2020-05-18T21:40:42.443 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2670bc8700 1 --2- 172.21.15.196:0/2615531915 >> v2:172.21.15.80:3300/0 conn(0x7f266c13bb70 0x7f266c13c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:42.443 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2670bc8700 1 --2- 172.21.15.196:0/2615531915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f266c0aae20 0x7f266c05c660 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.443 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2670bc8700 1 -- 172.21.15.196:0/2615531915 >> 172.21.15.196:0/2615531915 conn(0x7f266c061380 msgr2=0x7f266c069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:42.443 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2670bc8700 1 -- 172.21.15.196:0/2615531915 shutdown_connections 2020-05-18T21:40:42.443 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2670bc8700 1 -- 172.21.15.196:0/2615531915 wait complete. 2020-05-18T21:40:42.444 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2670bc8700 1 Processor -- start 2020-05-18T21:40:42.444 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2670bc8700 1 -- start start 2020-05-18T21:40:42.444 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2670bc8700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:42.444 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2670bc8700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266c135b60 con 0x7f266c0af080 2020-05-18T21:40:42.444 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2670bc8700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266c13a860 con 0x7f266c0aae20 2020-05-18T21:40:42.444 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2670bc8700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c0a7390 con 0x7f266c13bb70 2020-05-18T21:40:42.445 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f266b7fe700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f266c0aae20 0x7f266c0a2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43588/0 (socket says 172.21.15.196:43588) 2020-05-18T21:40:42.445 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f266b7fe700 1 -- 172.21.15.196:0/2219013573 learned_addr learned my addr 172.21.15.196:0/2219013573 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:42.445 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f266bfff700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.448 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2668ff9700 1 -- 172.21.15.196:0/2219013573 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594791328 0 0) 0x7f266c13a860 con 0x7f266c0aae20 2020-05-18T21:40:42.448 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2668ff9700 1 -- 172.21.15.196:0/2219013573 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2650002fe0 con 0x7f266c0aae20 2020-05-18T21:40:42.448 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2668ff9700 1 -- 172.21.15.196:0/2219013573 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293521332 0 0) 0x7f266c135b60 con 0x7f266c0af080 2020-05-18T21:40:42.448 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2668ff9700 1 -- 172.21.15.196:0/2219013573 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f266c13a860 con 0x7f266c0af080 2020-05-18T21:40:42.448 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f2668ff9700 1 -- 172.21.15.196:0/2219013573 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 1814616763 0 0) 0x7f2650002fe0 con 0x7f266c0aae20 2020-05-18T21:40:42.449 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f266bfff700 1 -- 172.21.15.196:0/2219013573 >> v1:172.21.15.196:6789/0 conn(0x7f266c0aae20 legacy=0x7f266c0a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.449 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f266bfff700 1 -- 172.21.15.196:0/2219013573 >> v1:172.21.15.80:6789/0 conn(0x7f266c0af080 legacy=0x7f266c132bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.449 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f266bfff700 1 -- 172.21.15.196:0/2219013573 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c135d90 con 0x7f266c13bb70 2020-05-18T21:40:42.449 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f266bfff700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f265c001110 tx=0x7f2664010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.449 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f266bfff700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f265c001110 tx=0x7f2664010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.450 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.439+0000 7f266bfff700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 secure :-1 s=READY pgs=30 cs=0 l=1 rx=0x7f265c001110 tx=0x7f2664010e80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:42.450 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f266bfff700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 secure :-1 s=THROTTLE_DONE pgs=30 cs=0 l=1 rx=0x7f265c001110 tx=0x7f2664010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.450 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f2668ff9700 1 -- 172.21.15.196:0/2219013573 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f265c001970 con 0x7f266c13bb70 2020-05-18T21:40:42.450 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f266bfff700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 secure :-1 s=THROTTLE_DONE pgs=30 cs=0 l=1 rx=0x7f265c001110 tx=0x7f2664010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.450 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f2668ff9700 1 -- 172.21.15.196:0/2219013573 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266400e370 con 0x7f266c13bb70 2020-05-18T21:40:42.450 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f266bfff700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 secure :-1 s=THROTTLE_DONE pgs=30 cs=0 l=1 rx=0x7f265c001110 tx=0x7f2664010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.451 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f2670bc8700 1 -- 172.21.15.196:0/2219013573 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266c0a32a0 con 0x7f266c13bb70 2020-05-18T21:40:42.451 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f2668ff9700 1 -- 172.21.15.196:0/2219013573 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2664005c70 con 0x7f266c13bb70 2020-05-18T21:40:42.451 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f2670bc8700 1 -- 172.21.15.196:0/2219013573 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266c13a5d0 con 0x7f266c13bb70 2020-05-18T21:40:42.451 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f266bfff700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 secure :-1 s=THROTTLE_DONE pgs=30 cs=0 l=1 rx=0x7f265c001110 tx=0x7f2664010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.451 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f2670bc8700 1 -- 172.21.15.196:0/2219013573 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266c0569e0 con 0x7f266c13bb70 2020-05-18T21:40:42.452 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f2668ff9700 1 -- 172.21.15.196:0/2219013573 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f266400c940 con 0x7f266c13bb70 2020-05-18T21:40:42.452 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f266bfff700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 secure :-1 s=THROTTLE_DONE pgs=30 cs=0 l=1 rx=0x7f265c001110 tx=0x7f2664010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.452 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f2668ff9700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f265004fe60 0x7f2650052320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:42.452 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f266b7fe700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f265004fe60 0x7f2650052320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.452 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f2668ff9700 1 -- 172.21.15.196:0/2219013573 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2664063900 con 0x7f266c13bb70 2020-05-18T21:40:42.453 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f266b7fe700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f265004fe60 0x7f2650052320 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f265c001210 tx=0x7f265c0095f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.453 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f266b7fe700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f265004fe60 0x7f2650052320 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f265c001210 tx=0x7f265c0095f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.453 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.443+0000 7f266b7fe700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f265004fe60 0x7f2650052320 secure :-1 s=READY pgs=72 cs=0 l=1 rx=0x7f265c001210 tx=0x7f265c0095f0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:42.454 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.455+0000 7f266bfff700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 secure :-1 s=THROTTLE_DONE pgs=30 cs=0 l=1 rx=0x7f265c001110 tx=0x7f2664010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.454 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.455+0000 7f2668ff9700 1 -- 172.21.15.196:0/2219013573 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f266403dcf0 con 0x7f266c13bb70 2020-05-18T21:40:42.570 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.571+0000 7f2670bc8700 1 -- 172.21.15.196:0/2219013573 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f266c05ac20 con 0x7f266c13bb70 2020-05-18T21:40:42.571 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.571+0000 7f266bfff700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 secure :-1 s=THROTTLE_DONE pgs=30 cs=0 l=1 rx=0x7f265c001110 tx=0x7f2664010e80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.571 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.571+0000 7f2668ff9700 1 -- 172.21.15.196:0/2219013573 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2664041310 con 0x7f266c13bb70 2020-05-18T21:40:42.572 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2020-05-18T21:40:42.573 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.571+0000 7f2670bc8700 1 -- 172.21.15.196:0/2219013573 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f265004fe60 msgr2=0x7f2650052320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.573 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.571+0000 7f2670bc8700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f265004fe60 0x7f2650052320 secure :-1 s=READY pgs=72 cs=0 l=1 rx=0x7f265c001210 tx=0x7f265c0095f0).stop 2020-05-18T21:40:42.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.571+0000 7f2670bc8700 1 -- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 msgr2=0x7f266c0a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.571+0000 7f2670bc8700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 secure :-1 s=READY pgs=30 cs=0 l=1 rx=0x7f265c001110 tx=0x7f2664010e80).stop 2020-05-18T21:40:42.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.575+0000 7f2670bc8700 1 -- 172.21.15.196:0/2219013573 shutdown_connections 2020-05-18T21:40:42.574 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.575+0000 7f2670bc8700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f265004fe60 0x7f2650052320 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.575 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.575+0000 7f2670bc8700 1 --2- 172.21.15.196:0/2219013573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f266c13bb70 0x7f266c0a6820 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.581 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.575+0000 7f2670bc8700 1 -- 172.21.15.196:0/2219013573 >> 172.21.15.196:0/2219013573 conn(0x7f266c061380 msgr2=0x7f266c0a8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:42.581 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.575+0000 7f2670bc8700 1 -- 172.21.15.196:0/2219013573 shutdown_connections 2020-05-18T21:40:42.581 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.575+0000 7f2670bc8700 1 -- 172.21.15.196:0/2219013573 wait complete. 2020-05-18T21:40:42.584 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-05-18T21:40:42.692 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.691+0000 7f334f59e700 1 Processor -- start 2020-05-18T21:40:42.692 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.691+0000 7f334f59e700 1 -- start start 2020-05-18T21:40:42.693 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.691+0000 7f334f59e700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f335005b070 0x7f33500a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:42.693 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.691+0000 7f334f59e700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a77c0 0x7f335013fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:42.694 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.691+0000 7f334f59e700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33500a3c80 con 0x7f33500a6c60 2020-05-18T21:40:42.694 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.691+0000 7f334f59e700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33500a3eb0 con 0x7f335005b070 2020-05-18T21:40:42.694 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.691+0000 7f334f59e700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335009ab30 con 0x7f33500a77c0 2020-05-18T21:40:42.695 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.691+0000 7f334ed9d700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a77c0 0x7f335013fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.695 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334ed9d700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a77c0 0x7f335013fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.196:38672/0 (socket says 172.21.15.196:38672) 2020-05-18T21:40:42.695 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334dd9b700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f33500a6c60 0x7f33500a70b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.196:39426/0 (socket says 172.21.15.196:39426) 2020-05-18T21:40:42.695 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334e59c700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f335005b070 0x7f33500a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.695 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334ed9d700 1 -- 172.21.15.196:0/2404559577 learned_addr learned my addr 172.21.15.196:0/2404559577 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:42.696 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334d59a700 1 -- 172.21.15.196:0/2404559577 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738342208 0 0) 0x7f33500a3c80 con 0x7f33500a6c60 2020-05-18T21:40:42.696 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334d59a700 1 -- 172.21.15.196:0/2404559577 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3334003270 con 0x7f33500a6c60 2020-05-18T21:40:42.699 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334ed9d700 1 -- 172.21.15.196:0/2404559577 >> v2:172.21.15.80:3300/0 conn(0x7f335005b070 msgr2=0x7f33500a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:42.700 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334ed9d700 1 --2- 172.21.15.196:0/2404559577 >> v2:172.21.15.80:3300/0 conn(0x7f335005b070 0x7f33500a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:42.700 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334ed9d700 1 -- 172.21.15.196:0/2404559577 >> v1:172.21.15.80:6789/0 conn(0x7f33500a6c60 legacy=0x7f33500a70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.700 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334ed9d700 1 -- 172.21.15.196:0/2404559577 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33500a25b0 con 0x7f33500a77c0 2020-05-18T21:40:42.700 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334e59c700 1 --2- 172.21.15.196:0/2404559577 >> v2:172.21.15.80:3300/0 conn(0x7f335005b070 0x7f33500a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_done state changed! 2020-05-18T21:40:42.700 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334ed9d700 1 --2- 172.21.15.196:0/2404559577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a77c0 0x7f335013fd50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f33480032f0 tx=0x7f3348007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.701 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334ed9d700 1 --2- 172.21.15.196:0/2404559577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a77c0 0x7f335013fd50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f33480032f0 tx=0x7f3348007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.701 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334ed9d700 1 --2- 172.21.15.196:0/2404559577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a77c0 0x7f335013fd50 secure :-1 s=READY pgs=31 cs=0 l=1 rx=0x7f33480032f0 tx=0x7f3348007e00).ready entity=mon.2 client_cookie=1156188c1174db8 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:42.701 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334ed9d700 1 --2- 172.21.15.196:0/2404559577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a77c0 0x7f335013fd50 secure :-1 s=THROTTLE_DONE pgs=31 cs=0 l=1 rx=0x7f33480032f0 tx=0x7f3348007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.701 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334d59a700 1 -- 172.21.15.196:0/2404559577 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3348013e90 con 0x7f33500a77c0 2020-05-18T21:40:42.701 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334ed9d700 1 --2- 172.21.15.196:0/2404559577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a77c0 0x7f335013fd50 secure :-1 s=THROTTLE_DONE pgs=31 cs=0 l=1 rx=0x7f33480032f0 tx=0x7f3348007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.702 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334ed9d700 1 --2- 172.21.15.196:0/2404559577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a77c0 0x7f335013fd50 secure :-1 s=THROTTLE_DONE pgs=31 cs=0 l=1 rx=0x7f33480032f0 tx=0x7f3348007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.702 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334d59a700 1 -- 172.21.15.196:0/2404559577 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334800c370 con 0x7f33500a77c0 2020-05-18T21:40:42.702 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334d59a700 1 -- 172.21.15.196:0/2404559577 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f334800c750 con 0x7f33500a77c0 2020-05-18T21:40:42.702 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334f59e700 1 -- 172.21.15.196:0/2404559577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a77c0 msgr2=0x7f335013fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.702 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334f59e700 1 --2- 172.21.15.196:0/2404559577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a77c0 0x7f335013fd50 secure :-1 s=READY pgs=31 cs=0 l=1 rx=0x7f33480032f0 tx=0x7f3348007e00).stop 2020-05-18T21:40:42.703 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334f59e700 1 -- 172.21.15.196:0/2404559577 shutdown_connections 2020-05-18T21:40:42.703 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334f59e700 1 --2- 172.21.15.196:0/2404559577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a77c0 0x7f335013fd50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.703 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334f59e700 1 --2- 172.21.15.196:0/2404559577 >> v2:172.21.15.80:3300/0 conn(0x7f335005b070 0x7f33500a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:42.703 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334f59e700 1 -- 172.21.15.196:0/2404559577 >> 172.21.15.196:0/2404559577 conn(0x7f3350060640 msgr2=0x7f335005a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:42.703 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334f59e700 1 -- 172.21.15.196:0/2404559577 shutdown_connections 2020-05-18T21:40:42.703 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334f59e700 1 -- 172.21.15.196:0/2404559577 wait complete. 2020-05-18T21:40:42.704 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334f59e700 1 Processor -- start 2020-05-18T21:40:42.704 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.695+0000 7f334f59e700 1 -- start start 2020-05-18T21:40:42.704 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334f59e700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f335005b070 0x7f335009d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:42.704 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334f59e700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a6c60 0x7f335008ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:42.704 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334f59e700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33500902b0 con 0x7f33500a77c0 2020-05-18T21:40:42.705 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334f59e700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3350091a90 con 0x7f335005b070 2020-05-18T21:40:42.705 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334f59e700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3350091d30 con 0x7f33500a6c60 2020-05-18T21:40:42.705 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f335005b070 0x7f335009d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.705 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f335005b070 0x7f335009d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34842/0 (socket says 172.21.15.196:34842) 2020-05-18T21:40:42.705 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 -- 172.21.15.196:0/1823157660 learned_addr learned my addr 172.21.15.196:0/1823157660 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:42.706 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334dd9b700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a6c60 0x7f335008ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.706 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 -- 172.21.15.196:0/1823157660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a6c60 msgr2=0x7f335008ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.706 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a6c60 0x7f335008ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.706 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 -- 172.21.15.196:0/1823157660 >> v1:172.21.15.80:6789/0 conn(0x7f33500a77c0 legacy=0x7f335008fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.706 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 -- 172.21.15.196:0/1823157660 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33501486f0 con 0x7f335005b070 2020-05-18T21:40:42.706 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 --2- 172.21.15.196:0/1823157660 >> v2:172.21.15.196:3300/0 conn(0x7f335005b070 0x7f335009d710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f3344002f30 tx=0x7f3344010ef0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.707 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 --2- 172.21.15.196:0/1823157660 >> v2:172.21.15.196:3300/0 conn(0x7f335005b070 0x7f335009d710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f3344002f30 tx=0x7f3344010ef0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.707 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f335005b070 0x7f335009d710 secure :-1 s=READY pgs=180 cs=0 l=1 rx=0x7f3344002f30 tx=0x7f3344010ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:42.707 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f335005b070 0x7f335009d710 secure :-1 s=THROTTLE_DONE pgs=180 cs=0 l=1 rx=0x7f3344002f30 tx=0x7f3344010ef0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.707 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f335005b070 0x7f335009d710 secure :-1 s=THROTTLE_DONE pgs=180 cs=0 l=1 rx=0x7f3344002f30 tx=0x7f3344010ef0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.707 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f333f7fe700 1 -- 172.21.15.196:0/1823157660 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3344005e30 con 0x7f335005b070 2020-05-18T21:40:42.708 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f335005b070 0x7f335009d710 secure :-1 s=THROTTLE_DONE pgs=180 cs=0 l=1 rx=0x7f3344002f30 tx=0x7f3344010ef0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.708 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f333f7fe700 1 -- 172.21.15.196:0/1823157660 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3344007180 con 0x7f335005b070 2020-05-18T21:40:42.708 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334f59e700 1 -- 172.21.15.196:0/1823157660 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33500916a0 con 0x7f335005b070 2020-05-18T21:40:42.708 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f333f7fe700 1 -- 172.21.15.196:0/1823157660 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33440075a0 con 0x7f335005b070 2020-05-18T21:40:42.708 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334f59e700 1 -- 172.21.15.196:0/1823157660 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335008f3e0 con 0x7f335005b070 2020-05-18T21:40:42.709 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334e59c700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f335005b070 0x7f335009d710 secure :-1 s=THROTTLE_DONE pgs=180 cs=0 l=1 rx=0x7f3344002f30 tx=0x7f3344010ef0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.710 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f334f59e700 1 -- 172.21.15.196:0/1823157660 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3350091e70 con 0x7f335005b070 2020-05-18T21:40:42.710 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.699+0000 7f333f7fe700 1 -- 172.21.15.196:0/1823157660 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f33440101a0 con 0x7f335005b070 2020-05-18T21:40:42.710 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.703+0000 7f334e59c700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f335005b070 0x7f335009d710 secure :-1 s=THROTTLE_DONE pgs=180 cs=0 l=1 rx=0x7f3344002f30 tx=0x7f3344010ef0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.710 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.703+0000 7f333f7fe700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f950 0x7f332c051e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:42.711 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.703+0000 7f333f7fe700 1 -- 172.21.15.196:0/1823157660 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3344063ec0 con 0x7f335005b070 2020-05-18T21:40:42.711 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.703+0000 7f334dd9b700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f950 0x7f332c051e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.711 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.703+0000 7f334dd9b700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f950 0x7f332c051e10 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f3340001110 tx=0x7f334000c890).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.711 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.703+0000 7f334dd9b700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f950 0x7f332c051e10 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f3340001110 tx=0x7f334000c890).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.711 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.703+0000 7f334dd9b700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f950 0x7f332c051e10 secure :-1 s=READY pgs=73 cs=0 l=1 rx=0x7f3340001110 tx=0x7f334000c890).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:42.712 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.707+0000 7f334e59c700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f335005b070 0x7f335009d710 secure :-1 s=THROTTLE_DONE pgs=180 cs=0 l=1 rx=0x7f3344002f30 tx=0x7f3344010ef0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.712 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.707+0000 7f333f7fe700 1 -- 172.21.15.196:0/1823157660 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f334403e250 con 0x7f335005b070 2020-05-18T21:40:42.820 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.819+0000 7f334f59e700 1 -- 172.21.15.196:0/1823157660 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f33500569e0 con 0x7f335005b070 2020-05-18T21:40:42.821 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.823+0000 7f334e59c700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f335005b070 0x7f335009d710 secure :-1 s=THROTTLE_DONE pgs=180 cs=0 l=1 rx=0x7f3344002f30 tx=0x7f3344010ef0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.822 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.823+0000 7f333f7fe700 1 -- 172.21.15.196:0/1823157660 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3344041870 con 0x7f335005b070 2020-05-18T21:40:42.822 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 8 2020-05-18T21:40:42.823 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.823+0000 7f334f59e700 1 -- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f950 msgr2=0x7f332c051e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.823+0000 7f334f59e700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f950 0x7f332c051e10 secure :-1 s=READY pgs=73 cs=0 l=1 rx=0x7f3340001110 tx=0x7f334000c890).stop 2020-05-18T21:40:42.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.823+0000 7f334f59e700 1 -- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f335005b070 msgr2=0x7f335009d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.823+0000 7f334f59e700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f335005b070 0x7f335009d710 secure :-1 s=READY pgs=180 cs=0 l=1 rx=0x7f3344002f30 tx=0x7f3344010ef0).stop 2020-05-18T21:40:42.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.823+0000 7f334f59e700 1 -- 172.21.15.196:0/1823157660 shutdown_connections 2020-05-18T21:40:42.824 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.823+0000 7f334f59e700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f332c04f950 0x7f332c051e10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.823+0000 7f334f59e700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f33500a6c60 0x7f335008ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.823+0000 7f334f59e700 1 --2- 172.21.15.196:0/1823157660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f335005b070 0x7f335009d710 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.823+0000 7f334f59e700 1 -- 172.21.15.196:0/1823157660 >> 172.21.15.196:0/1823157660 conn(0x7f3350060640 msgr2=0x7f33500a4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.823+0000 7f334f59e700 1 -- 172.21.15.196:0/1823157660 shutdown_connections 2020-05-18T21:40:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.823+0000 7f334f59e700 1 -- 172.21.15.196:0/1823157660 wait complete. 2020-05-18T21:40:42.835 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-05-18T21:40:42.945 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086f24c700 1 Processor -- start 2020-05-18T21:40:42.946 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086f24c700 1 -- start start 2020-05-18T21:40:42.947 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086f24c700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f086813bec0 0x7f08680a6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:42.947 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086f24c700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0868134b80 0x7f0868135010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:42.947 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086f24c700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08680a55e0 con 0x7f0868069e00 2020-05-18T21:40:42.947 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086f24c700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08680a5880 con 0x7f0868134b80 2020-05-18T21:40:42.948 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086f24c700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680a5b20 con 0x7f086813bec0 2020-05-18T21:40:42.948 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086e24a700 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0868069e00 0x7f086813b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.196:39438/0 (socket says 172.21.15.196:39438) 2020-05-18T21:40:42.948 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086e24a700 1 -- 172.21.15.196:0/1766215529 learned_addr learned my addr 172.21.15.196:0/1766215529 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:42.948 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086da49700 1 --2- 172.21.15.196:0/1766215529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f086813bec0 0x7f08680a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.949 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086ea4b700 1 --2- 172.21.15.196:0/1766215529 >> v2:172.21.15.80:3300/0 conn(0x7f0868134b80 0x7f0868135010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.953 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086d248700 1 -- 172.21.15.196:0/1766215529 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562200724 0 0) 0x7f08680a55e0 con 0x7f0868069e00 2020-05-18T21:40:42.953 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086d248700 1 -- 172.21.15.196:0/1766215529 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0850003270 con 0x7f0868069e00 2020-05-18T21:40:42.953 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086da49700 1 -- 172.21.15.196:0/1766215529 >> v2:172.21.15.80:3300/0 conn(0x7f0868134b80 msgr2=0x7f0868135010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:42.953 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086da49700 1 --2- 172.21.15.196:0/1766215529 >> v2:172.21.15.80:3300/0 conn(0x7f0868134b80 0x7f0868135010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:42.954 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086da49700 1 -- 172.21.15.196:0/1766215529 >> v1:172.21.15.80:6789/0 conn(0x7f0868069e00 legacy=0x7f086813b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.954 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086da49700 1 -- 172.21.15.196:0/1766215529 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08680a6d70 con 0x7f086813bec0 2020-05-18T21:40:42.954 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086da49700 1 --2- 172.21.15.196:0/1766215529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f086813bec0 0x7f08680a6910 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f0858002f70 tx=0x7f0858007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.954 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086da49700 1 --2- 172.21.15.196:0/1766215529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f086813bec0 0x7f08680a6910 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f0858002f70 tx=0x7f0858007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.954 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086da49700 1 --2- 172.21.15.196:0/1766215529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f086813bec0 0x7f08680a6910 secure :-1 s=READY pgs=32 cs=0 l=1 rx=0x7f0858002f70 tx=0x7f0858007a80).ready entity=mon.2 client_cookie=e3d228b1659611fc server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:42.955 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086da49700 1 --2- 172.21.15.196:0/1766215529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f086813bec0 0x7f08680a6910 secure :-1 s=THROTTLE_DONE pgs=32 cs=0 l=1 rx=0x7f0858002f70 tx=0x7f0858007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.955 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086d248700 1 -- 172.21.15.196:0/1766215529 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0858008e80 con 0x7f086813bec0 2020-05-18T21:40:42.955 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086da49700 1 --2- 172.21.15.196:0/1766215529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f086813bec0 0x7f08680a6910 secure :-1 s=THROTTLE_DONE pgs=32 cs=0 l=1 rx=0x7f0858002f70 tx=0x7f0858007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.955 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086d248700 1 -- 172.21.15.196:0/1766215529 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0858011540 con 0x7f086813bec0 2020-05-18T21:40:42.955 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086da49700 1 --2- 172.21.15.196:0/1766215529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f086813bec0 0x7f08680a6910 secure :-1 s=THROTTLE_DONE pgs=32 cs=0 l=1 rx=0x7f0858002f70 tx=0x7f0858007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.956 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086d248700 1 -- 172.21.15.196:0/1766215529 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0858011920 con 0x7f086813bec0 2020-05-18T21:40:42.956 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086f24c700 1 -- 172.21.15.196:0/1766215529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f086813bec0 msgr2=0x7f08680a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.956 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.947+0000 7f086f24c700 1 --2- 172.21.15.196:0/1766215529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f086813bec0 0x7f08680a6910 secure :-1 s=READY pgs=32 cs=0 l=1 rx=0x7f0858002f70 tx=0x7f0858007a80).stop 2020-05-18T21:40:42.956 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 -- 172.21.15.196:0/1766215529 shutdown_connections 2020-05-18T21:40:42.956 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 --2- 172.21.15.196:0/1766215529 >> v2:172.21.15.80:3300/0 conn(0x7f0868134b80 0x7f0868135010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:42.957 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 --2- 172.21.15.196:0/1766215529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f086813bec0 0x7f08680a6910 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.957 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 -- 172.21.15.196:0/1766215529 >> 172.21.15.196:0/1766215529 conn(0x7f086805b290 msgr2=0x7f08680aad10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:42.957 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 -- 172.21.15.196:0/1766215529 shutdown_connections 2020-05-18T21:40:42.957 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 -- 172.21.15.196:0/1766215529 wait complete. 2020-05-18T21:40:42.957 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 Processor -- start 2020-05-18T21:40:42.957 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 -- start start 2020-05-18T21:40:42.958 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0868069e00 0x7f086813a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:42.958 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f086813bec0 0x7f086814fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:42.958 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08680a9130 con 0x7f0868134b80 2020-05-18T21:40:42.958 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0868131190 con 0x7f0868069e00 2020-05-18T21:40:42.958 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f086812d050 con 0x7f086813bec0 2020-05-18T21:40:42.959 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086ea4b700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f086813bec0 0x7f086814fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.959 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086ea4b700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f086813bec0 0x7f086814fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34860/0 (socket says 172.21.15.196:34860) 2020-05-18T21:40:42.959 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086ea4b700 1 -- 172.21.15.196:0/3719722310 learned_addr learned my addr 172.21.15.196:0/3719722310 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:42.959 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086e24a700 1 --2- 172.21.15.196:0/3719722310 >> v2:172.21.15.80:3300/0 conn(0x7f0868069e00 0x7f086813a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f08577fe700 1 -- 172.21.15.196:0/3719722310 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993754758 0 0) 0x7f08680a9130 con 0x7f0868134b80 2020-05-18T21:40:42.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f08577fe700 1 -- 172.21.15.196:0/3719722310 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0848002fe0 con 0x7f0868134b80 2020-05-18T21:40:42.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086ea4b700 1 -- 172.21.15.196:0/3719722310 >> v2:172.21.15.80:3300/0 conn(0x7f0868069e00 msgr2=0x7f086813a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.960 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086ea4b700 1 --2- 172.21.15.196:0/3719722310 >> v2:172.21.15.80:3300/0 conn(0x7f0868069e00 0x7f086813a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:42.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086ea4b700 1 -- 172.21.15.196:0/3719722310 >> v1:172.21.15.80:6789/0 conn(0x7f0868134b80 legacy=0x7f0868131d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:42.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086ea4b700 1 -- 172.21.15.196:0/3719722310 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868133480 con 0x7f086813bec0 2020-05-18T21:40:42.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086ea4b700 1 --2- 172.21.15.196:0/3719722310 >> v2:172.21.15.196:3300/0 conn(0x7f086813bec0 0x7f086814fb10 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f08600032f0 tx=0x7f0860010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.961 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086ea4b700 1 --2- 172.21.15.196:0/3719722310 >> v2:172.21.15.196:3300/0 conn(0x7f086813bec0 0x7f086814fb10 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f08600032f0 tx=0x7f0860010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.962 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086ea4b700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f086813bec0 0x7f086814fb10 secure :-1 s=READY pgs=181 cs=0 l=1 rx=0x7f08600032f0 tx=0x7f0860010f70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:42.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086ea4b700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f086813bec0 0x7f086814fb10 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f08600032f0 tx=0x7f0860010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f08577fe700 1 -- 172.21.15.196:0/3719722310 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0860005a80 con 0x7f086813bec0 2020-05-18T21:40:42.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086ea4b700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f086813bec0 0x7f086814fb10 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f08600032f0 tx=0x7f0860010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086ea4b700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f086813bec0 0x7f086814fb10 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f08600032f0 tx=0x7f0860010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.963 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 -- 172.21.15.196:0/3719722310 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086812d2f0 con 0x7f086813bec0 2020-05-18T21:40:42.964 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f08577fe700 1 -- 172.21.15.196:0/3719722310 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0860005be0 con 0x7f086813bec0 2020-05-18T21:40:42.964 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f08577fe700 1 -- 172.21.15.196:0/3719722310 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0860012860 con 0x7f086813bec0 2020-05-18T21:40:42.964 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.951+0000 7f086f24c700 1 -- 172.21.15.196:0/3719722310 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08680aa8b0 con 0x7f086813bec0 2020-05-18T21:40:42.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.955+0000 7f086ea4b700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f086813bec0 0x7f086814fb10 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f08600032f0 tx=0x7f0860010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.955+0000 7f086f24c700 1 -- 172.21.15.196:0/3719722310 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08680569e0 con 0x7f086813bec0 2020-05-18T21:40:42.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.955+0000 7f08577fe700 1 -- 172.21.15.196:0/3719722310 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f0860012a90 con 0x7f086813bec0 2020-05-18T21:40:42.966 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.955+0000 7f086ea4b700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f086813bec0 0x7f086814fb10 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f08600032f0 tx=0x7f0860010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.955+0000 7f08577fe700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f084804fd80 0x7f0848052240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:42.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.955+0000 7f086e24a700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f084804fd80 0x7f0848052240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:42.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.955+0000 7f08577fe700 1 -- 172.21.15.196:0/3719722310 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0860063ab0 con 0x7f086813bec0 2020-05-18T21:40:42.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.955+0000 7f086e24a700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f084804fd80 0x7f0848052240 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0864001110 tx=0x7f0864001d80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.967 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.955+0000 7f086e24a700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f084804fd80 0x7f0848052240 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0864001110 tx=0x7f0864001d80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.968 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.955+0000 7f086e24a700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f084804fd80 0x7f0848052240 secure :-1 s=READY pgs=74 cs=0 l=1 rx=0x7f0864001110 tx=0x7f0864001d80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:42.968 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.959+0000 7f086ea4b700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f086813bec0 0x7f086814fb10 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f08600032f0 tx=0x7f0860010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:42.968 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:42.963+0000 7f08577fe700 1 -- 172.21.15.196:0/3719722310 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f086003de40 con 0x7f086813bec0 2020-05-18T21:40:43.075 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.075+0000 7f086f24c700 1 -- 172.21.15.196:0/3719722310 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08680abb90 con 0x7f086813bec0 2020-05-18T21:40:43.076 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.075+0000 7f086ea4b700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f086813bec0 0x7f086814fb10 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f08600032f0 tx=0x7f0860010f70).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.076 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.075+0000 7f08577fe700 1 -- 172.21.15.196:0/3719722310 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0860041460 con 0x7f086813bec0 2020-05-18T21:40:43.076 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 8 2020-05-18T21:40:43.079 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.079+0000 7f086f24c700 1 -- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f084804fd80 msgr2=0x7f0848052240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.079 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.079+0000 7f086f24c700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f084804fd80 0x7f0848052240 secure :-1 s=READY pgs=74 cs=0 l=1 rx=0x7f0864001110 tx=0x7f0864001d80).stop 2020-05-18T21:40:43.079 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.079+0000 7f086f24c700 1 -- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f086813bec0 msgr2=0x7f086814fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.079 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.079+0000 7f086f24c700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f086813bec0 0x7f086814fb10 secure :-1 s=READY pgs=181 cs=0 l=1 rx=0x7f08600032f0 tx=0x7f0860010f70).stop 2020-05-18T21:40:43.080 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.079+0000 7f086f24c700 1 -- 172.21.15.196:0/3719722310 shutdown_connections 2020-05-18T21:40:43.080 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.079+0000 7f086f24c700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f084804fd80 0x7f0848052240 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:43.080 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.079+0000 7f086f24c700 1 --2- 172.21.15.196:0/3719722310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f086813bec0 0x7f086814fb10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:43.080 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.079+0000 7f086f24c700 1 --2- 172.21.15.196:0/3719722310 >> v2:172.21.15.80:3300/0 conn(0x7f0868069e00 0x7f086813a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:43.080 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.079+0000 7f086f24c700 1 -- 172.21.15.196:0/3719722310 >> 172.21.15.196:0/3719722310 conn(0x7f086805b290 msgr2=0x7f08680a7760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:43.081 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.079+0000 7f086f24c700 1 -- 172.21.15.196:0/3719722310 shutdown_connections 2020-05-18T21:40:43.081 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.079+0000 7f086f24c700 1 -- 172.21.15.196:0/3719722310 wait complete. 2020-05-18T21:40:43.088 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-05-18T21:40:43.194 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2f0b3700 1 Processor -- start 2020-05-18T21:40:43.194 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2f0b3700 1 -- start start 2020-05-18T21:40:43.197 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2f0b3700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbf28066680 0x7fbf2813c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:43.197 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2f0b3700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf28140d00 0x7fbf281430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:43.197 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2f0b3700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf280aaa90 con 0x7fbf2805bd40 2020-05-18T21:40:43.197 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2f0b3700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf280a88c0 con 0x7fbf28066680 2020-05-18T21:40:43.197 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2f0b3700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf280aad30 con 0x7fbf28140d00 2020-05-18T21:40:43.198 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2ce4f700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbf28066680 0x7fbf2813c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:43.198 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2ce4f700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbf28066680 0x7fbf2813c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34864/0 (socket says 172.21.15.196:34864) 2020-05-18T21:40:43.198 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2ce4f700 1 -- 172.21.15.196:0/3043142065 learned_addr learned my addr 172.21.15.196:0/3043142065 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:43.198 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2d650700 1 --2- 172.21.15.196:0/3043142065 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf28140d00 0x7fbf281430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:43.198 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2ce4f700 1 -- 172.21.15.196:0/3043142065 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf28140d00 msgr2=0x7fbf281430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:43.199 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2ce4f700 1 --2- 172.21.15.196:0/3043142065 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf28140d00 0x7fbf281430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:43.199 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2ce4f700 1 -- 172.21.15.196:0/3043142065 >> v1:172.21.15.80:6789/0 conn(0x7fbf2805bd40 legacy=0x7fbf2805c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.199 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2ce4f700 1 -- 172.21.15.196:0/3043142065 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf280a1e00 con 0x7fbf28066680 2020-05-18T21:40:43.199 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2d650700 1 --2- 172.21.15.196:0/3043142065 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf28140d00 0x7fbf281430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request state changed! 2020-05-18T21:40:43.199 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2ce4f700 1 --2- 172.21.15.196:0/3043142065 >> v2:172.21.15.196:3300/0 conn(0x7fbf28066680 0x7fbf2813c290 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fbf18002f70 tx=0x7fbf18007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2ce4f700 1 --2- 172.21.15.196:0/3043142065 >> v2:172.21.15.196:3300/0 conn(0x7fbf28066680 0x7fbf2813c290 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fbf18002f70 tx=0x7fbf18007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2ce4f700 1 --2- 172.21.15.196:0/3043142065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf28066680 0x7fbf2813c290 secure :-1 s=READY pgs=182 cs=0 l=1 rx=0x7fbf18002f70 tx=0x7fbf18007a80).ready entity=mon.1 client_cookie=34efbd8efb4db1ef server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:43.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2ce4f700 1 --2- 172.21.15.196:0/3043142065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf28066680 0x7fbf2813c290 secure :-1 s=THROTTLE_DONE pgs=182 cs=0 l=1 rx=0x7fbf18002f70 tx=0x7fbf18007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf277fe700 1 -- 172.21.15.196:0/3043142065 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf18011760 con 0x7fbf28066680 2020-05-18T21:40:43.200 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2ce4f700 1 --2- 172.21.15.196:0/3043142065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf28066680 0x7fbf2813c290 secure :-1 s=THROTTLE_DONE pgs=182 cs=0 l=1 rx=0x7fbf18002f70 tx=0x7fbf18007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf277fe700 1 -- 172.21.15.196:0/3043142065 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf180118c0 con 0x7fbf28066680 2020-05-18T21:40:43.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2ce4f700 1 --2- 172.21.15.196:0/3043142065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf28066680 0x7fbf2813c290 secure :-1 s=THROTTLE_DONE pgs=182 cs=0 l=1 rx=0x7fbf18002f70 tx=0x7fbf18007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf277fe700 1 -- 172.21.15.196:0/3043142065 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf18011ce0 con 0x7fbf28066680 2020-05-18T21:40:43.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/3043142065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf28066680 msgr2=0x7fbf2813c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.201 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.195+0000 7fbf2f0b3700 1 --2- 172.21.15.196:0/3043142065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf28066680 0x7fbf2813c290 secure :-1 s=READY pgs=182 cs=0 l=1 rx=0x7fbf18002f70 tx=0x7fbf18007a80).stop 2020-05-18T21:40:43.205 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/3043142065 shutdown_connections 2020-05-18T21:40:43.205 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2f0b3700 1 --2- 172.21.15.196:0/3043142065 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf28140d00 0x7fbf281430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:43.205 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2f0b3700 1 --2- 172.21.15.196:0/3043142065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf28066680 0x7fbf2813c290 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:43.205 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/3043142065 >> 172.21.15.196:0/3043142065 conn(0x7fbf28069ac0 msgr2=0x7fbf28065c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:43.206 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/3043142065 shutdown_connections 2020-05-18T21:40:43.206 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/3043142065 wait complete. 2020-05-18T21:40:43.206 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2f0b3700 1 Processor -- start 2020-05-18T21:40:43.206 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2f0b3700 1 -- start start 2020-05-18T21:40:43.206 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2f0b3700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbf28140d00 0x7fbf280a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:43.207 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2f0b3700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf280a4910 con 0x7fbf28066680 2020-05-18T21:40:43.207 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2f0b3700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf280a4b40 con 0x7fbf2805bd40 2020-05-18T21:40:43.207 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2f0b3700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf281499c0 con 0x7fbf28140d00 2020-05-18T21:40:43.207 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2ce4f700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fbf2805bd40 0x7fbf281492b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43630/0 (socket says 172.21.15.196:43630) 2020-05-18T21:40:43.207 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2ce4f700 1 -- 172.21.15.196:0/348696127 learned_addr learned my addr 172.21.15.196:0/348696127 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:43.208 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf2d650700 1 --2- 172.21.15.196:0/348696127 >> v2:172.21.15.80:3300/0 conn(0x7fbf28140d00 0x7fbf280a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:43.208 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf25ffb700 1 -- 172.21.15.196:0/348696127 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257686952 0 0) 0x7fbf280a4b40 con 0x7fbf2805bd40 2020-05-18T21:40:43.208 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.199+0000 7fbf25ffb700 1 -- 172.21.15.196:0/348696127 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf08002fe0 con 0x7fbf2805bd40 2020-05-18T21:40:43.208 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf25ffb700 1 -- 172.21.15.196:0/348696127 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 4017293150 0 0) 0x7fbf08002fe0 con 0x7fbf2805bd40 2020-05-18T21:40:43.208 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2d650700 1 -- 172.21.15.196:0/348696127 >> v1:172.21.15.196:6789/0 conn(0x7fbf2805bd40 legacy=0x7fbf281492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.209 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2d650700 1 -- 172.21.15.196:0/348696127 >> v1:172.21.15.80:6789/0 conn(0x7fbf28066680 legacy=0x7fbf2814ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.209 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2d650700 1 -- 172.21.15.196:0/348696127 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf280a2560 con 0x7fbf28140d00 2020-05-18T21:40:43.209 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2d650700 1 --2- 172.21.15.196:0/348696127 >> v2:172.21.15.80:3300/0 conn(0x7fbf28140d00 0x7fbf280a4fe0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fbf1c001020 tx=0x7fbf20010d30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.209 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2d650700 1 --2- 172.21.15.196:0/348696127 >> v2:172.21.15.80:3300/0 conn(0x7fbf28140d00 0x7fbf280a4fe0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fbf1c001020 tx=0x7fbf20010d30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.209 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2d650700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf28140d00 0x7fbf280a4fe0 secure :-1 s=READY pgs=135 cs=0 l=1 rx=0x7fbf1c001020 tx=0x7fbf20010d30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:43.210 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2d650700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf28140d00 0x7fbf280a4fe0 secure :-1 s=THROTTLE_DONE pgs=135 cs=0 l=1 rx=0x7fbf1c001020 tx=0x7fbf20010d30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.210 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf25ffb700 1 -- 172.21.15.196:0/348696127 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf18008600 con 0x7fbf28140d00 2020-05-18T21:40:43.210 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2d650700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf28140d00 0x7fbf280a4fe0 secure :-1 s=THROTTLE_DONE pgs=135 cs=0 l=1 rx=0x7fbf1c001020 tx=0x7fbf20010d30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.210 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/348696127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf280a1b90 con 0x7fbf28140d00 2020-05-18T21:40:43.210 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2d650700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf28140d00 0x7fbf280a4fe0 secure :-1 s=THROTTLE_DONE pgs=135 cs=0 l=1 rx=0x7fbf1c001020 tx=0x7fbf20010d30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.211 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf25ffb700 1 -- 172.21.15.196:0/348696127 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf20012930 con 0x7fbf28140d00 2020-05-18T21:40:43.211 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf25ffb700 1 -- 172.21.15.196:0/348696127 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf20012c90 con 0x7fbf28140d00 2020-05-18T21:40:43.211 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/348696127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2813b640 con 0x7fbf28140d00 2020-05-18T21:40:43.211 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2d650700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf28140d00 0x7fbf280a4fe0 secure :-1 s=THROTTLE_DONE pgs=135 cs=0 l=1 rx=0x7fbf1c001020 tx=0x7fbf20010d30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.211 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/348696127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2813e420 con 0x7fbf28140d00 2020-05-18T21:40:43.211 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf25ffb700 1 -- 172.21.15.196:0/348696127 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fbf20013070 con 0x7fbf28140d00 2020-05-18T21:40:43.212 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2d650700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf28140d00 0x7fbf280a4fe0 secure :-1 s=THROTTLE_DONE pgs=135 cs=0 l=1 rx=0x7fbf1c001020 tx=0x7fbf20010d30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.212 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf25ffb700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbf0804fce0 0x7fbf080521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:43.212 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf25ffb700 1 -- 172.21.15.196:0/348696127 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbf20063810 con 0x7fbf28140d00 2020-05-18T21:40:43.212 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2ce4f700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbf0804fce0 0x7fbf080521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:43.212 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.203+0000 7fbf2ce4f700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbf0804fce0 0x7fbf080521a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fbf18010340 tx=0x7fbf18007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.213 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.207+0000 7fbf2ce4f700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbf0804fce0 0x7fbf080521a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fbf18010340 tx=0x7fbf18007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.213 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.207+0000 7fbf2ce4f700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbf0804fce0 0x7fbf080521a0 secure :-1 s=READY pgs=75 cs=0 l=1 rx=0x7fbf18010340 tx=0x7fbf18007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:43.213 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.215+0000 7fbf2d650700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf28140d00 0x7fbf280a4fe0 secure :-1 s=THROTTLE_DONE pgs=135 cs=0 l=1 rx=0x7fbf1c001020 tx=0x7fbf20010d30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.214 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.215+0000 7fbf25ffb700 1 -- 172.21.15.196:0/348696127 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7fbf2813e420 con 0x7fbf28140d00 2020-05-18T21:40:43.326 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.327+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/348696127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf2813e420 con 0x7fbf28140d00 2020-05-18T21:40:43.327 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.327+0000 7fbf2d650700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf28140d00 0x7fbf280a4fe0 secure :-1 s=THROTTLE_DONE pgs=135 cs=0 l=1 rx=0x7fbf1c001020 tx=0x7fbf20010d30).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.327+0000 7fbf25ffb700 1 -- 172.21.15.196:0/348696127 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbf2813e420 con 0x7fbf28140d00 2020-05-18T21:40:43.328 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 8 2020-05-18T21:40:43.330 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.331+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/348696127 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbf0804fce0 msgr2=0x7fbf080521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.330 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.331+0000 7fbf2f0b3700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbf0804fce0 0x7fbf080521a0 secure :-1 s=READY pgs=75 cs=0 l=1 rx=0x7fbf18010340 tx=0x7fbf18007a80).stop 2020-05-18T21:40:43.330 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.331+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/348696127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf28140d00 msgr2=0x7fbf280a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.330 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.331+0000 7fbf2f0b3700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf28140d00 0x7fbf280a4fe0 secure :-1 s=READY pgs=135 cs=0 l=1 rx=0x7fbf1c001020 tx=0x7fbf20010d30).stop 2020-05-18T21:40:43.331 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.331+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/348696127 shutdown_connections 2020-05-18T21:40:43.331 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.331+0000 7fbf2f0b3700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fbf0804fce0 0x7fbf080521a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:43.331 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.331+0000 7fbf2f0b3700 1 --2- 172.21.15.196:0/348696127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf28140d00 0x7fbf280a4fe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:43.331 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.331+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/348696127 >> 172.21.15.196:0/348696127 conn(0x7fbf28069ac0 msgr2=0x7fbf2805eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:43.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.331+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/348696127 shutdown_connections 2020-05-18T21:40:43.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.331+0000 7fbf2f0b3700 1 -- 172.21.15.196:0/348696127 wait complete. 2020-05-18T21:40:43.341 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-05-18T21:40:43.440 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.439+0000 7f4c3d4a7700 1 Processor -- start 2020-05-18T21:40:43.441 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.439+0000 7f4c3d4a7700 1 -- start start 2020-05-18T21:40:43.443 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4c38065f70 0x7f4c380aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:43.444 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4c38066910 0x7f4c3805a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:43.444 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3805ae30 0x7f4c3813b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:43.444 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c380a9d60 con 0x7f4c38065f70 2020-05-18T21:40:43.444 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3805b2e0 con 0x7f4c38066910 2020-05-18T21:40:43.444 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c380a9360 con 0x7f4c3805ae30 2020-05-18T21:40:43.445 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4c38066910 0x7f4c3805a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:43.445 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4c38066910 0x7f4c3805a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34882/0 (socket says 172.21.15.196:34882) 2020-05-18T21:40:43.445 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 -- 172.21.15.196:0/719219482 learned_addr learned my addr 172.21.15.196:0/719219482 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:43.445 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c37fff700 1 --2- 172.21.15.196:0/719219482 >> v2:172.21.15.80:3300/0 conn(0x7f4c38065f70 0x7f4c380aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:43.445 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3cca6700 1 --2- 172.21.15.196:0/719219482 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3805ae30 0x7f4c3813b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:43.446 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 -- 172.21.15.196:0/719219482 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3805ae30 msgr2=0x7f4c3813b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:43.446 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 --2- 172.21.15.196:0/719219482 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3805ae30 0x7f4c3813b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:43.446 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 -- 172.21.15.196:0/719219482 >> v2:172.21.15.80:3300/0 conn(0x7f4c38065f70 msgr2=0x7f4c380aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:43.446 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 --2- 172.21.15.196:0/719219482 >> v2:172.21.15.80:3300/0 conn(0x7f4c38065f70 0x7f4c380aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:43.446 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 -- 172.21.15.196:0/719219482 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c380aab80 con 0x7f4c38066910 2020-05-18T21:40:43.447 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 --2- 172.21.15.196:0/719219482 >> v2:172.21.15.196:3300/0 conn(0x7f4c38066910 0x7f4c3805a9d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f4c380aaec0 tx=0x7f4c28007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.447 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 --2- 172.21.15.196:0/719219482 >> v2:172.21.15.196:3300/0 conn(0x7f4c38066910 0x7f4c3805a9d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f4c380aaec0 tx=0x7f4c28007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.447 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 --2- 172.21.15.196:0/719219482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c38066910 0x7f4c3805a9d0 secure :-1 s=READY pgs=184 cs=0 l=1 rx=0x7f4c380aaec0 tx=0x7f4c28007a80).ready entity=mon.1 client_cookie=809702da721eb075 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:43.447 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 --2- 172.21.15.196:0/719219482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c38066910 0x7f4c3805a9d0 secure :-1 s=THROTTLE_DONE pgs=184 cs=0 l=1 rx=0x7f4c380aaec0 tx=0x7f4c28007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.447 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c36ffd700 1 -- 172.21.15.196:0/719219482 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c28011720 con 0x7f4c38066910 2020-05-18T21:40:43.448 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 --2- 172.21.15.196:0/719219482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c38066910 0x7f4c3805a9d0 secure :-1 s=THROTTLE_DONE pgs=184 cs=0 l=1 rx=0x7f4c380aaec0 tx=0x7f4c28007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.448 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c36ffd700 1 -- 172.21.15.196:0/719219482 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c28011880 con 0x7f4c38066910 2020-05-18T21:40:43.448 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c377fe700 1 --2- 172.21.15.196:0/719219482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c38066910 0x7f4c3805a9d0 secure :-1 s=THROTTLE_DONE pgs=184 cs=0 l=1 rx=0x7f4c380aaec0 tx=0x7f4c28007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.448 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/719219482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c38066910 msgr2=0x7f4c3805a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.448 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c36ffd700 1 -- 172.21.15.196:0/719219482 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c28011ca0 con 0x7f4c38066910 2020-05-18T21:40:43.449 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 --2- 172.21.15.196:0/719219482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c38066910 0x7f4c3805a9d0 secure :-1 s=READY pgs=184 cs=0 l=1 rx=0x7f4c380aaec0 tx=0x7f4c28007a80).stop 2020-05-18T21:40:43.449 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/719219482 shutdown_connections 2020-05-18T21:40:43.449 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 --2- 172.21.15.196:0/719219482 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3805ae30 0x7f4c3813b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:43.449 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 --2- 172.21.15.196:0/719219482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c38066910 0x7f4c3805a9d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:43.450 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 --2- 172.21.15.196:0/719219482 >> v2:172.21.15.80:3300/0 conn(0x7f4c38065f70 0x7f4c380aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:43.450 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/719219482 >> 172.21.15.196:0/719219482 conn(0x7f4c380ae900 msgr2=0x7f4c380aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:43.450 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/719219482 shutdown_connections 2020-05-18T21:40:43.450 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/719219482 wait complete. 2020-05-18T21:40:43.455 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 Processor -- start 2020-05-18T21:40:43.455 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.443+0000 7f4c3d4a7700 1 -- start start 2020-05-18T21:40:43.455 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c3d4a7700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c38065f70 0x7f4c380a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:43.455 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c3d4a7700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3809e6c0 con 0x7f4c3805ae30 2020-05-18T21:40:43.456 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c3d4a7700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3809e960 con 0x7f4c38066910 2020-05-18T21:40:43.456 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c3d4a7700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c380a57a0 con 0x7f4c38065f70 2020-05-18T21:40:43.456 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c3cca6700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f4c38066910 0x7f4c38145570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43646/0 (socket says 172.21.15.196:43646) 2020-05-18T21:40:43.456 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c3cca6700 1 -- 172.21.15.196:0/3966574947 learned_addr learned my addr 172.21.15.196:0/3966574947 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:43.456 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c377fe700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c38065f70 0x7f4c380a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:43.457 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284050752 0 0) 0x7f4c3809e960 con 0x7f4c38066910 2020-05-18T21:40:43.457 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c18002fe0 con 0x7f4c38066910 2020-05-18T21:40:43.457 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 935+0+0 (unknown 4167243389 0 0) 0x7f4c18002fe0 con 0x7f4c38066910 2020-05-18T21:40:43.457 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 --> v1:172.21.15.196:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4c3809e960 con 0x7f4c38066910 2020-05-18T21:40:43.458 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7f4c30001920 con 0x7f4c38066910 2020-05-18T21:40:43.458 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c38065f70 msgr2=0x7f4c380a77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.458 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c38065f70 0x7f4c380a77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:43.458 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 >> v1:172.21.15.196:6789/0 conn(0x7f4c38066910 legacy=0x7f4c38145570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.458 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 >> v1:172.21.15.80:6789/0 conn(0x7f4c3805ae30 legacy=0x7f4c380a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.459 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:43.459 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c18005310 0x7f4c18003a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:43.459 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c38065f70 0x7f4c18003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:43.459 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c380a57a0 con 0x7f4c18005310 2020-05-18T21:40:43.459 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c18004490 con 0x7f4c18004a80 2020-05-18T21:40:43.459 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c18004650 con 0x7f4c38065f70 2020-05-18T21:40:43.460 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c37fff700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:43.460 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c377fe700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c18005310 0x7f4c18003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:43.460 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c3cca6700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c38065f70 0x7f4c18003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:43.460 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c37fff700 1 -- 172.21.15.196:0/3966574947 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c38065f70 msgr2=0x7f4c18003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.460 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c37fff700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c38065f70 0x7f4c18003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c37fff700 1 -- 172.21.15.196:0/3966574947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c18005310 msgr2=0x7f4c18003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c37fff700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c18005310 0x7f4c18003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c37fff700 1 -- 172.21.15.196:0/3966574947 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3813be70 con 0x7f4c18004a80 2020-05-18T21:40:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/3966574947 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3809e3b0 con 0x7f4c18004a80 2020-05-18T21:40:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/3966574947 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c380a1a40 con 0x7f4c18004a80 2020-05-18T21:40:43.462 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c37fff700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4c2c0030b0 tx=0x7f4c2c010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.462 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c37fff700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4c2c0030b0 tx=0x7f4c2c010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.462 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c37fff700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 secure :-1 s=READY pgs=186 cs=0 l=1 rx=0x7f4c2c0030b0 tx=0x7f4c2c010cf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:43.462 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c37fff700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 secure :-1 s=THROTTLE_DONE pgs=186 cs=0 l=1 rx=0x7f4c2c0030b0 tx=0x7f4c2c010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.462 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c37fff700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 secure :-1 s=THROTTLE_DONE pgs=186 cs=0 l=1 rx=0x7f4c2c0030b0 tx=0x7f4c2c010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.463 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c37fff700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 secure :-1 s=THROTTLE_DONE pgs=186 cs=0 l=1 rx=0x7f4c2c0030b0 tx=0x7f4c2c010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.463 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c2c005ae0 con 0x7f4c18004a80 2020-05-18T21:40:43.463 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c005c40 con 0x7f4c18004a80 2020-05-18T21:40:43.463 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c2c005e70 con 0x7f4c18004a80 2020-05-18T21:40:43.463 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c37fff700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 secure :-1 s=THROTTLE_DONE pgs=186 cs=0 l=1 rx=0x7f4c2c0030b0 tx=0x7f4c2c010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.464 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.447+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/3966574947 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c14003380 con 0x7f4c18004a80 2020-05-18T21:40:43.464 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.451+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f4c2c0120d0 con 0x7f4c18004a80 2020-05-18T21:40:43.464 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.451+0000 7f4c37fff700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 secure :-1 s=THROTTLE_DONE pgs=186 cs=0 l=1 rx=0x7f4c2c0030b0 tx=0x7f4c2c010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.464 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.451+0000 7f4c357fa700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f4c18057b70 0x7f4c1805a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:43.465 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.451+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c2c063df0 con 0x7f4c18004a80 2020-05-18T21:40:43.465 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.451+0000 7f4c377fe700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f4c18057b70 0x7f4c1805a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:43.465 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.451+0000 7f4c377fe700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f4c18057b70 0x7f4c1805a030 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4c380aaec0 tx=0x7f4c28007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.465 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.451+0000 7f4c377fe700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f4c18057b70 0x7f4c1805a030 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4c380aaec0 tx=0x7f4c28007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.466 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.451+0000 7f4c377fe700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f4c18057b70 0x7f4c1805a030 secure :-1 s=READY pgs=76 cs=0 l=1 rx=0x7f4c380aaec0 tx=0x7f4c28007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:43.466 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.455+0000 7f4c37fff700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 secure :-1 s=THROTTLE_DONE pgs=186 cs=0 l=1 rx=0x7f4c2c0030b0 tx=0x7f4c2c010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.466 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.455+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f4c2c067e60 con 0x7f4c18004a80 2020-05-18T21:40:43.600 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.599+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/3966574947 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c14003720 con 0x7f4c18004a80 2020-05-18T21:40:43.601 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.603+0000 7f4c37fff700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 secure :-1 s=THROTTLE_DONE pgs=186 cs=0 l=1 rx=0x7f4c2c0030b0 tx=0x7f4c2c010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:43.601 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.603+0000 7f4c357fa700 1 -- 172.21.15.196:0/3966574947 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2131 (secure 0 0 0) 0x7f4c2c03e1e0 con 0x7f4c18004a80 2020-05-18T21:40:43.602 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:43.602 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.196:6835/993135284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":993135284},{"type":"v1","addr":"172.21.15.196:6835","nonce":993135284}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":7}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-05-18T21:40:32.208271+0000","modified":"2020-05-18T21:40:36.226347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4386":{"gid":4386,"name":"b","rank":0,"incarnation":5,"state":"up:active","state_seq":2,"addr":"172.21.15.80:6833/386820289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":386820289},{"type":"v1","addr":"172.21.15.80:6833","nonce":386820289}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-05-18T21:40:43.603 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.603+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f4c18057b70 msgr2=0x7f4c1805a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.603 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.603+0000 7f4c3d4a7700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f4c18057b70 0x7f4c1805a030 secure :-1 s=READY pgs=76 cs=0 l=1 rx=0x7f4c380aaec0 tx=0x7f4c28007a80).stop 2020-05-18T21:40:43.603 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.603+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 msgr2=0x7f4c18004e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.603+0000 7f4c3d4a7700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 secure :-1 s=READY pgs=186 cs=0 l=1 rx=0x7f4c2c0030b0 tx=0x7f4c2c010cf0).stop 2020-05-18T21:40:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.603+0000 7f4c3cca6700 1 -- 172.21.15.196:0/3966574947 reap_dead start 2020-05-18T21:40:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.603+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/3966574947 shutdown_connections 2020-05-18T21:40:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.603+0000 7f4c3d4a7700 1 --2- 172.21.15.196:0/3966574947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c18004a80 0x7f4c18004e60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.603+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/3966574947 >> 172.21.15.196:0/3966574947 conn(0x7f4c380ae900 msgr2=0x7f4c380ab6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.603+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/3966574947 shutdown_connections 2020-05-18T21:40:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:43.603+0000 7f4c3d4a7700 1 -- 172.21.15.196:0/3966574947 wait complete. 2020-05-18T21:40:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 7 2020-05-18T21:40:43.815 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-05-18T21:40:32.208271+0000', 'modified': '2020-05-18T21:40:36.226347+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4386}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4386': {'gid': 4386, 'name': 'b', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.80:6833/386820289', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6832', 'nonce': 386820289}, {'type': 'v1', 'addr': '172.21.15.80:6833', 'nonce': 386820289}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-05-18T21:40:43.815 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-05-18T21:40:43.816 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2020-05-18T21:40:43.931 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2020-05-18T21:40:43.931 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "d048673a-3d91-4cc6-882d-e9cc70e7cd6c", 2020-05-18T21:40:43.931 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2020-05-18T21:40:43.931 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 4386, 2020-05-18T21:40:43.932 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2020-05-18T21:40:43.932 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2020-05-18T21:40:43.932 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 8.7017248580000004, 2020-05-18T21:40:43.932 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 6, 2020-05-18T21:40:43.932 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 22, 2020-05-18T21:40:43.933 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 22, 2020-05-18T21:40:43.933 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 9.3542428270000002 2020-05-18T21:40:43.933 INFO:teuthology.orchestra.run.smithi080.stdout:} 2020-05-18T21:40:43.940 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "d048673a-3d91-4cc6-882d-e9cc70e7cd6c", "whoami": 0, "id": 4386, "want_state": "up:active", "state": "up:active", "rank_uptime": 8.7017248580000004, "mdsmap_epoch": 6, "osdmap_epoch": 22, "osdmap_epoch_barrier": 22, "uptime": 9.3542428270000002 } 2020-05-18T21:40:43.940 INFO:teuthology.run_tasks:Running task ceph-fuse... 2020-05-18T21:40:43.969 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2020-05-18T21:40:43.970 INFO:tasks.ceph_fuse:config is None 2020-05-18T21:40:43.970 INFO:tasks.ceph_fuse:new config is {'client.0': None} 2020-05-18T21:40:43.971 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2020-05-18T21:40:43.971 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-05-18T21:40:44.080 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.079+0000 7f1b0caaf700 1 Processor -- start 2020-05-18T21:40:44.080 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.079+0000 7f1b0caaf700 1 -- start start 2020-05-18T21:40:44.084 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.079+0000 7f1b0caaf700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1b08140840 0x7f1b08147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:44.084 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.079+0000 7f1b0caaf700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0809e0b0 con 0x7f1b080a7220 2020-05-18T21:40:44.084 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.079+0000 7f1b0caaf700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b080a20f0 con 0x7f1b080a4080 2020-05-18T21:40:44.084 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.079+0000 7f1b0caaf700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b080a3580 con 0x7f1b08140840 2020-05-18T21:40:44.084 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f1b080a4080 0x7f1b08140130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43658/0 (socket says 172.21.15.196:43658) 2020-05-18T21:40:44.085 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 -- 172.21.15.196:0/676385995 learned_addr learned my addr 172.21.15.196:0/676385995 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:44.085 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b07fff700 1 --2- 172.21.15.196:0/676385995 >> v2:172.21.15.80:3300/0 conn(0x7f1b08140840 0x7f1b08147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.085 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 549651844 0 0) 0x7f1b080a20f0 con 0x7f1b080a4080 2020-05-18T21:40:44.085 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af4003270 con 0x7f1b080a4080 2020-05-18T21:40:44.085 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212810046 0 0) 0x7f1b0809e0b0 con 0x7f1b080a7220 2020-05-18T21:40:44.086 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b080a20f0 con 0x7f1b080a7220 2020-05-18T21:40:44.086 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4165265265 0 0) 0x7f1af4003270 con 0x7f1b080a4080 2020-05-18T21:40:44.086 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 >> v2:172.21.15.80:3300/0 conn(0x7f1b08140840 msgr2=0x7f1b08147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:44.086 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 --2- 172.21.15.196:0/676385995 >> v2:172.21.15.80:3300/0 conn(0x7f1b08140840 0x7f1b08147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:44.086 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 >> v1:172.21.15.80:6789/0 conn(0x7f1b080a7220 legacy=0x7f1b080a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b080a6a60 con 0x7f1b080a4080 2020-05-18T21:40:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1918888239 0 0) 0x7f1b00001a90 con 0x7f1b080a4080 2020-05-18T21:40:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 >> v1:172.21.15.196:6789/0 conn(0x7f1b080a4080 legacy=0x7f1b08140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af4003940 0x7f1af4003d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:44.087 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af4004510 0x7f1af4004990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:44.088 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1af4004df0 0x7f1af400b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:44.088 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af400bbc0 con 0x7f1af4004df0 2020-05-18T21:40:44.088 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af400bd00 con 0x7f1af4004510 2020-05-18T21:40:44.088 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af400be40 con 0x7f1af4003940 2020-05-18T21:40:44.088 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af4004510 0x7f1af4004990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.089 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b077fe700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af4003940 0x7f1af4003d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.089 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b07fff700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1af4004df0 0x7f1af400b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.089 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 -- 172.21.15.196:0/676385995 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af4003940 msgr2=0x7f1af4003d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:44.089 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af4003940 0x7f1af4003d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:44.089 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 -- 172.21.15.196:0/676385995 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1af4004df0 msgr2=0x7f1af400b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:44.089 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1af4004df0 0x7f1af400b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:44.090 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b07fff700 1 -- 172.21.15.196:0/676385995 reap_dead start 2020-05-18T21:40:44.090 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 -- 172.21.15.196:0/676385995 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af4003270 con 0x7f1af4004510 2020-05-18T21:40:44.090 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af4004510 0x7f1af4004990 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.090 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af4004510 0x7f1af4004990 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.090 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af4004510 0x7f1af4004990 secure :-1 s=READY pgs=188 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).ready entity=mon.1 client_cookie=2ef4e85165ad8f86 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:44.091 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af4004510 0x7f1af4004990 secure :-1 s=THROTTLE_DONE pgs=188 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.091 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af4004510 0x7f1af4004990 secure :-1 s=THROTTLE_DONE pgs=188 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.091 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b00005e20 con 0x7f1af4004510 2020-05-18T21:40:44.091 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af4004510 0x7f1af4004990 secure :-1 s=THROTTLE_DONE pgs=188 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.091 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b00004c70 con 0x7f1af4004510 2020-05-18T21:40:44.092 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b0caaf700 1 -- 172.21.15.196:0/676385995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af4004510 msgr2=0x7f1af4004990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.092 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b067fc700 1 -- 172.21.15.196:0/676385995 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b00004fd0 con 0x7f1af4004510 2020-05-18T21:40:44.092 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b0caaf700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af4004510 0x7f1af4004990 secure :-1 s=READY pgs=188 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).stop 2020-05-18T21:40:44.095 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b0caaf700 1 -- 172.21.15.196:0/676385995 shutdown_connections 2020-05-18T21:40:44.096 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b0caaf700 1 --2- 172.21.15.196:0/676385995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af4004510 0x7f1af4004990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.096 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b0caaf700 1 -- 172.21.15.196:0/676385995 >> 172.21.15.196:0/676385995 conn(0x7f1b0805a7c0 msgr2=0x7f1b0805abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:44.096 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b0caaf700 1 -- 172.21.15.196:0/676385995 shutdown_connections 2020-05-18T21:40:44.096 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.083+0000 7f1b0caaf700 1 -- 172.21.15.196:0/676385995 wait complete. 2020-05-18T21:40:44.097 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b0caaf700 1 Processor -- start 2020-05-18T21:40:44.097 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b0caaf700 1 -- start start 2020-05-18T21:40:44.097 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b0caaf700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1b080982b0 0x7f1b0809cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:44.097 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b0caaf700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1b08149d10 0x7f1b080989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:44.097 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b0caaf700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0805b2e0 con 0x7f1af4004510 2020-05-18T21:40:44.098 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b0caaf700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0809d0c0 con 0x7f1b08149d10 2020-05-18T21:40:44.098 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b0caaf700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0814a420 con 0x7f1b080982b0 2020-05-18T21:40:44.098 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1b080982b0 0x7f1b0809cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.098 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1b080982b0 0x7f1b0809cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34910/0 (socket says 172.21.15.196:34910) 2020-05-18T21:40:44.098 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 -- 172.21.15.196:0/1120320707 learned_addr learned my addr 172.21.15.196:0/1120320707 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:44.099 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b07fff700 1 --2- 172.21.15.196:0/1120320707 >> v2:172.21.15.80:3300/0 conn(0x7f1b08149d10 0x7f1b080989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.099 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 -- 172.21.15.196:0/1120320707 >> v2:172.21.15.80:3300/0 conn(0x7f1b08149d10 msgr2=0x7f1b080989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.099 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/1120320707 >> v2:172.21.15.80:3300/0 conn(0x7f1b08149d10 0x7f1b080989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.099 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b04ff9700 1 -- 172.21.15.196:0/1120320707 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853746862 0 0) 0x7f1b0805b2e0 con 0x7f1af4004510 2020-05-18T21:40:44.099 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 -- 172.21.15.196:0/1120320707 >> v1:172.21.15.80:6789/0 conn(0x7f1af4004510 legacy=0x7f1b08097ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 -- 172.21.15.196:0/1120320707 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0814a6c0 con 0x7f1b080982b0 2020-05-18T21:40:44.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/1120320707 >> v2:172.21.15.196:3300/0 conn(0x7f1b080982b0 0x7f1b0809cc60 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/1120320707 >> v2:172.21.15.196:3300/0 conn(0x7f1b080982b0 0x7f1b0809cc60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b080982b0 0x7f1b0809cc60 secure :-1 s=READY pgs=189 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:44.101 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b080982b0 0x7f1b0809cc60 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.101 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b04ff9700 1 -- 172.21.15.196:0/1120320707 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b000118e0 con 0x7f1b080982b0 2020-05-18T21:40:44.101 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b080982b0 0x7f1b0809cc60 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.101 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b04ff9700 1 -- 172.21.15.196:0/1120320707 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b00011a40 con 0x7f1b080982b0 2020-05-18T21:40:44.101 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b080982b0 0x7f1b0809cc60 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.102 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b0caaf700 1 -- 172.21.15.196:0/1120320707 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b08097700 con 0x7f1b080982b0 2020-05-18T21:40:44.102 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b04ff9700 1 -- 172.21.15.196:0/1120320707 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b0000e180 con 0x7f1b080982b0 2020-05-18T21:40:44.102 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b0caaf700 1 -- 172.21.15.196:0/1120320707 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b08098e30 con 0x7f1b080982b0 2020-05-18T21:40:44.102 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.087+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b080982b0 0x7f1b0809cc60 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.102 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.091+0000 7f1b04ff9700 1 -- 172.21.15.196:0/1120320707 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f1b0000e380 con 0x7f1b080982b0 2020-05-18T21:40:44.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.091+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b080982b0 0x7f1b0809cc60 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.091+0000 7f1b0caaf700 1 -- 172.21.15.196:0/1120320707 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b08097890 con 0x7f1b080982b0 2020-05-18T21:40:44.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.091+0000 7f1b04ff9700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1ae804f850 0x7f1ae8051d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:44.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.091+0000 7f1b04ff9700 1 -- 172.21.15.196:0/1120320707 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1b00011e40 con 0x7f1b080982b0 2020-05-18T21:40:44.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.091+0000 7f1b077fe700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1ae804f850 0x7f1ae8051d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.091+0000 7f1b077fe700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1ae804f850 0x7f1ae8051d10 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1afc0012f0 tx=0x7f1afc00c910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.091+0000 7f1b077fe700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1ae804f850 0x7f1ae8051d10 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1afc0012f0 tx=0x7f1afc00c910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.091+0000 7f1b077fe700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1ae804f850 0x7f1ae8051d10 secure :-1 s=READY pgs=77 cs=0 l=1 rx=0x7f1afc0012f0 tx=0x7f1afc00c910).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:44.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.095+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b080982b0 0x7f1b0809cc60 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.095+0000 7f1b04ff9700 1 -- 172.21.15.196:0/1120320707 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f1b0003e8c0 con 0x7f1b080982b0 2020-05-18T21:40:44.210 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.211+0000 7f1b0caaf700 1 -- 172.21.15.196:0/1120320707 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b080569e0 con 0x7f1b080982b0 2020-05-18T21:40:44.211 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.211+0000 7f1b06ffd700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b080982b0 0x7f1b0809cc60 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.212 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.211+0000 7f1b04ff9700 1 -- 172.21.15.196:0/1120320707 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16089 (secure 0 0 0) 0x7f1b00041ee0 con 0x7f1b080982b0 2020-05-18T21:40:44.212 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:44.212 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":22,"fsid":"d048673a-3d91-4cc6-882d-e9cc70e7cd6c","created":"2020-05-18T21:40:15.911738+0000","modified":"2020-05-18T21:40:33.211283+0000","last_up_change":"2020-05-18T21:40:22.762927+0000","last_in_change":"2020-05-18T21:40:22.762927+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-05-18T21:40:19.808941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-05-18T21:40:26.944311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-05-18T21:40:30.135661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-05-18T21:40:31.183488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8db63a0a-61dc-42b5-9559-fddcf436cab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":12771},{"type":"v1","addr":"172.21.15.196:6801","nonce":12771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":12771},{"type":"v1","addr":"172.21.15.196:6803","nonce":12771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":12771},{"type":"v1","addr":"172.21.15.196:6807","nonce":12771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":12771},{"type":"v1","addr":"172.21.15.196:6805","nonce":12771}]},"public_addr":"172.21.15.196:6801/12771","cluster_addr":"172.21.15.196:6803/12771","heartbeat_back_addr":"172.21.15.196:6807/12771","heartbeat_front_addr":"172.21.15.196:6805/12771","state":["exists","up"]},{"osd":1,"uuid":"6773b0c3-9f49-42a2-b75b-c10c734ff086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":12772},{"type":"v1","addr":"172.21.15.196:6817","nonce":12772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":12772},{"type":"v1","addr":"172.21.15.196:6819","nonce":12772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":12772},{"type":"v1","addr":"172.21.15.196:6823","nonce":12772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":12772},{"type":"v1","addr":"172.21.15.196:6821","nonce":12772}]},"public_addr":"172.21.15.196:6817/12772","cluster_addr":"172.21.15.196:6819/12772","heartbeat_back_addr":"172.21.15.196:6823/12772","heartbeat_front_addr":"172.21.15.196:6821/12772","state":["exists","up"]},{"osd":2,"uuid":"ac1f2e05-9bb7-4768-b712-10e4d2ea4154","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":12773},{"type":"v1","addr":"172.21.15.196:6809","nonce":12773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":12773},{"type":"v1","addr":"172.21.15.196:6811","nonce":12773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":12773},{"type":"v1","addr":"172.21.15.196:6815","nonce":12773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":12773},{"type":"v1","addr":"172.21.15.196:6813","nonce":12773}]},"public_addr":"172.21.15.196:6809/12773","cluster_addr":"172.21.15.196:6811/12773","heartbeat_back_addr":"172.21.15.196:6815/12773","heartbeat_front_addr":"172.21.15.196:6813/12773","state":["exists","up"]},{"osd":3,"uuid":"bae5ab40-235b-4e0f-8d0d-524afd099b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":12775},{"type":"v1","addr":"172.21.15.196:6825","nonce":12775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":12775},{"type":"v1","addr":"172.21.15.196:6827","nonce":12775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":12775},{"type":"v1","addr":"172.21.15.196:6831","nonce":12775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":12775},{"type":"v1","addr":"172.21.15.196:6829","nonce":12775}]},"public_addr":"172.21.15.196:6825/12775","cluster_addr":"172.21.15.196:6827/12775","heartbeat_back_addr":"172.21.15.196:6831/12775","heartbeat_front_addr":"172.21.15.196:6829/12775","state":["exists","up"]},{"osd":4,"uuid":"275fc52c-58e4-4f35-a0cb-e117fe36bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":13040},{"type":"v1","addr":"172.21.15.80:6819","nonce":13040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6821","nonce":13040},{"type":"v1","addr":"172.21.15.80:6823","nonce":13040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6829","nonce":13040},{"type":"v1","addr":"172.21.15.80:6831","nonce":13040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":13040},{"type":"v1","addr":"172.21.15.80:6827","nonce":13040}]},"public_addr":"172.21.15.80:6819/13040","cluster_addr":"172.21.15.80:6823/13040","heartbeat_back_addr":"172.21.15.80:6831/13040","heartbeat_front_addr":"172.21.15.80:6827/13040","state":["exists","up"]},{"osd":5,"uuid":"9795b89c-e2f8-4abf-b9c7-796727a57002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":13037},{"type":"v1","addr":"172.21.15.80:6818","nonce":13037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":13037},{"type":"v1","addr":"172.21.15.80:6822","nonce":13037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":13037},{"type":"v1","addr":"172.21.15.80:6830","nonce":13037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":13037},{"type":"v1","addr":"172.21.15.80:6826","nonce":13037}]},"public_addr":"172.21.15.80:6818/13037","cluster_addr":"172.21.15.80:6822/13037","heartbeat_back_addr":"172.21.15.80:6830/13037","heartbeat_front_addr":"172.21.15.80:6826/13037","state":["exists","up"]},{"osd":6,"uuid":"e42df2e8-9cc5-4c7e-a243-bd18ccfbd002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":13039},{"type":"v1","addr":"172.21.15.80:6804","nonce":13039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":13039},{"type":"v1","addr":"172.21.15.80:6808","nonce":13039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":13039},{"type":"v1","addr":"172.21.15.80:6813","nonce":13039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":13039},{"type":"v1","addr":"172.21.15.80:6810","nonce":13039}]},"public_addr":"172.21.15.80:6804/13039","cluster_addr":"172.21.15.80:6808/13039","heartbeat_back_addr":"172.21.15.80:6813/13039","heartbeat_front_addr":"172.21.15.80:6810/13039","state":["exists","up"]},{"osd":7,"uuid":"ebd6c938-beff-4668-b70e-076ed22bb8c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":13038},{"type":"v1","addr":"172.21.15.80:6801","nonce":13038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":13038},{"type":"v1","addr":"172.21.15.80:6805","nonce":13038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":13038},{"type":"v1","addr":"172.21.15.80:6817","nonce":13038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":13038},{"type":"v1","addr":"172.21.15.80:6811","nonce":13038}]},"public_addr":"172.21.15.80:6801/13038","cluster_addr":"172.21.15.80:6805/13038","heartbeat_back_addr":"172.21.15.80:6817/13038","heartbeat_front_addr":"172.21.15.80:6811/13038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.134515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.439836+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.476825+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.176642+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.452301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.427183+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.230250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-05-18T21:40:21.210168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-18T21:40:44.213 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.211+0000 7f1b0caaf700 1 -- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1ae804f850 msgr2=0x7f1ae8051d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.214 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.211+0000 7f1b0caaf700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1ae804f850 0x7f1ae8051d10 secure :-1 s=READY pgs=77 cs=0 l=1 rx=0x7f1afc0012f0 tx=0x7f1afc00c910).stop 2020-05-18T21:40:44.214 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.215+0000 7f1b0caaf700 1 -- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b080982b0 msgr2=0x7f1b0809cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.214 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.215+0000 7f1b0caaf700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b080982b0 0x7f1b0809cc60 secure :-1 s=READY pgs=189 cs=0 l=1 rx=0x7f1b00001860 tx=0x7f1b080a7cb0).stop 2020-05-18T21:40:44.214 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.215+0000 7f1b0caaf700 1 -- 172.21.15.196:0/1120320707 shutdown_connections 2020-05-18T21:40:44.215 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.215+0000 7f1b0caaf700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f1ae804f850 0x7f1ae8051d10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.215 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.215+0000 7f1b0caaf700 1 --2- 172.21.15.196:0/1120320707 >> v2:172.21.15.80:3300/0 conn(0x7f1b08149d10 0x7f1b080989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.215 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.215+0000 7f1b0caaf700 1 --2- 172.21.15.196:0/1120320707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b080982b0 0x7f1b0809cc60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.215 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.215+0000 7f1b0caaf700 1 -- 172.21.15.196:0/1120320707 >> 172.21.15.196:0/1120320707 conn(0x7f1b0805a7c0 msgr2=0x7f1b0805eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:44.215 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.215+0000 7f1b0caaf700 1 -- 172.21.15.196:0/1120320707 shutdown_connections 2020-05-18T21:40:44.215 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.215+0000 7f1b0caaf700 1 -- 172.21.15.196:0/1120320707 wait complete. 2020-05-18T21:40:44.225 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-05-18T21:40:19.808941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-05-18T21:40:26.944311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-05-18T21:40:30.135661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-05-18T21:40:31.183488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-05-18T21:40:44.226 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-05-18T21:40:44.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.327+0000 7f31620f6700 1 Processor -- start 2020-05-18T21:40:44.328 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.327+0000 7f31620f6700 1 -- start start 2020-05-18T21:40:44.330 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f31620f6700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f315c0ab7c0 0x7f315c13bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:44.331 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f31620f6700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0a8720 0x7f315c13c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:44.331 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f31620f6700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f315c0aa9a0 con 0x7f315c065c30 2020-05-18T21:40:44.331 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f31620f6700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f315c0a94a0 con 0x7f315c0ab7c0 2020-05-18T21:40:44.331 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f31620f6700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315c0a9740 con 0x7f315c0a8720 2020-05-18T21:40:44.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315b7fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f315c0ab7c0 0x7f315c13bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315b7fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f315c0ab7c0 0x7f315c13bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34916/0 (socket says 172.21.15.196:34916) 2020-05-18T21:40:44.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315b7fe700 1 -- 172.21.15.196:0/4152504632 learned_addr learned my addr 172.21.15.196:0/4152504632 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:44.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315affd700 1 --2- 172.21.15.196:0/4152504632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0a8720 0x7f315c13c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315b7fe700 1 -- 172.21.15.196:0/4152504632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0a8720 msgr2=0x7f315c13c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:44.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315b7fe700 1 --2- 172.21.15.196:0/4152504632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0a8720 0x7f315c13c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:44.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315b7fe700 1 -- 172.21.15.196:0/4152504632 >> v1:172.21.15.80:6789/0 conn(0x7f315c065c30 legacy=0x7f315c0660a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315b7fe700 1 -- 172.21.15.196:0/4152504632 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315c0a99e0 con 0x7f315c0ab7c0 2020-05-18T21:40:44.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315affd700 1 --2- 172.21.15.196:0/4152504632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0a8720 0x7f315c13c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_reply_more state changed! 2020-05-18T21:40:44.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315b7fe700 1 --2- 172.21.15.196:0/4152504632 >> v2:172.21.15.196:3300/0 conn(0x7f315c0ab7c0 0x7f315c13bcd0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f3144002f70 tx=0x7f3144007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315b7fe700 1 --2- 172.21.15.196:0/4152504632 >> v2:172.21.15.196:3300/0 conn(0x7f315c0ab7c0 0x7f315c13bcd0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f3144002f70 tx=0x7f3144007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315b7fe700 1 --2- 172.21.15.196:0/4152504632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f315c0ab7c0 0x7f315c13bcd0 secure :-1 s=READY pgs=190 cs=0 l=1 rx=0x7f3144002f70 tx=0x7f3144007a80).ready entity=mon.1 client_cookie=1d9a0b55e254a8e0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:44.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315b7fe700 1 --2- 172.21.15.196:0/4152504632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f315c0ab7c0 0x7f315c13bcd0 secure :-1 s=THROTTLE_DONE pgs=190 cs=0 l=1 rx=0x7f3144002f70 tx=0x7f3144007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315a7fc700 1 -- 172.21.15.196:0/4152504632 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3144011760 con 0x7f315c0ab7c0 2020-05-18T21:40:44.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315b7fe700 1 --2- 172.21.15.196:0/4152504632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f315c0ab7c0 0x7f315c13bcd0 secure :-1 s=THROTTLE_DONE pgs=190 cs=0 l=1 rx=0x7f3144002f70 tx=0x7f3144007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315b7fe700 1 --2- 172.21.15.196:0/4152504632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f315c0ab7c0 0x7f315c13bcd0 secure :-1 s=THROTTLE_DONE pgs=190 cs=0 l=1 rx=0x7f3144002f70 tx=0x7f3144007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315a7fc700 1 -- 172.21.15.196:0/4152504632 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31440118c0 con 0x7f315c0ab7c0 2020-05-18T21:40:44.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f315a7fc700 1 -- 172.21.15.196:0/4152504632 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3144011ce0 con 0x7f315c0ab7c0 2020-05-18T21:40:44.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f31620f6700 1 -- 172.21.15.196:0/4152504632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f315c0ab7c0 msgr2=0x7f315c13bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f31620f6700 1 --2- 172.21.15.196:0/4152504632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f315c0ab7c0 0x7f315c13bcd0 secure :-1 s=READY pgs=190 cs=0 l=1 rx=0x7f3144002f70 tx=0x7f3144007a80).stop 2020-05-18T21:40:44.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f31620f6700 1 -- 172.21.15.196:0/4152504632 shutdown_connections 2020-05-18T21:40:44.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f31620f6700 1 --2- 172.21.15.196:0/4152504632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0a8720 0x7f315c13c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:44.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f31620f6700 1 --2- 172.21.15.196:0/4152504632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f315c0ab7c0 0x7f315c13bcd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.347 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f31620f6700 1 -- 172.21.15.196:0/4152504632 >> 172.21.15.196:0/4152504632 conn(0x7f315c069ac0 msgr2=0x7f315c069ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:44.347 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f31620f6700 1 -- 172.21.15.196:0/4152504632 shutdown_connections 2020-05-18T21:40:44.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.331+0000 7f31620f6700 1 -- 172.21.15.196:0/4152504632 wait complete. 2020-05-18T21:40:44.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f31620f6700 1 Processor -- start 2020-05-18T21:40:44.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f31620f6700 1 -- start start 2020-05-18T21:40:44.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f31620f6700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f315c065c30 0x7f315c1379c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:44.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f31620f6700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:44.349 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f31620f6700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f315c13ac60 con 0x7f315c0a8720 2020-05-18T21:40:44.349 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f31620f6700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f315c13a280 con 0x7f315c065c30 2020-05-18T21:40:44.349 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f31620f6700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315c13a520 con 0x7f315c0ab7c0 2020-05-18T21:40:44.349 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315b7fe700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f315c065c30 0x7f315c1379c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.349 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315bfff700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.350 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315bfff700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.196:38758/0 (socket says 172.21.15.196:38758) 2020-05-18T21:40:44.350 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315bfff700 1 -- 172.21.15.196:0/864374457 learned_addr learned my addr 172.21.15.196:0/864374457 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:44.350 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315bfff700 1 -- 172.21.15.196:0/864374457 >> v2:172.21.15.80:3300/0 conn(0x7f315c065c30 msgr2=0x7f315c1379c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.350 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315bfff700 1 --2- 172.21.15.196:0/864374457 >> v2:172.21.15.80:3300/0 conn(0x7f315c065c30 0x7f315c1379c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.350 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315bfff700 1 -- 172.21.15.196:0/864374457 >> v1:172.21.15.80:6789/0 conn(0x7f315c0a8720 legacy=0x7f315c0a50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315bfff700 1 -- 172.21.15.196:0/864374457 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315c13a7c0 con 0x7f315c0ab7c0 2020-05-18T21:40:44.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315bfff700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f315c060390 tx=0x7f315400c7a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315bfff700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f315c060390 tx=0x7f315400c7a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315bfff700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 secure :-1 s=READY pgs=33 cs=0 l=1 rx=0x7f315c060390 tx=0x7f315400c7a0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:44.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315bfff700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 secure :-1 s=THROTTLE_DONE pgs=33 cs=0 l=1 rx=0x7f315c060390 tx=0x7f315400c7a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.351 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f3158ff9700 1 -- 172.21.15.196:0/864374457 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31540101e0 con 0x7f315c0ab7c0 2020-05-18T21:40:44.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315bfff700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 secure :-1 s=THROTTLE_DONE pgs=33 cs=0 l=1 rx=0x7f315c060390 tx=0x7f315400c7a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f3158ff9700 1 -- 172.21.15.196:0/864374457 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3154010340 con 0x7f315c0ab7c0 2020-05-18T21:40:44.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f315bfff700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 secure :-1 s=THROTTLE_DONE pgs=33 cs=0 l=1 rx=0x7f315c060390 tx=0x7f315400c7a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f31620f6700 1 -- 172.21.15.196:0/864374457 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f315c0a57d0 con 0x7f315c0ab7c0 2020-05-18T21:40:44.352 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f3158ff9700 1 -- 172.21.15.196:0/864374457 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3154010760 con 0x7f315c0ab7c0 2020-05-18T21:40:44.353 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.335+0000 7f31620f6700 1 -- 172.21.15.196:0/864374457 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f315c0a2ed0 con 0x7f315c0ab7c0 2020-05-18T21:40:44.353 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.339+0000 7f315bfff700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 secure :-1 s=THROTTLE_DONE pgs=33 cs=0 l=1 rx=0x7f315c060390 tx=0x7f315400c7a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.353 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.339+0000 7f31620f6700 1 -- 172.21.15.196:0/864374457 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f315c13a9c0 con 0x7f315c0ab7c0 2020-05-18T21:40:44.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.339+0000 7f315bfff700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 secure :-1 s=THROTTLE_DONE pgs=33 cs=0 l=1 rx=0x7f315c060390 tx=0x7f315400c7a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.339+0000 7f3158ff9700 1 -- 172.21.15.196:0/864374457 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f31540068a0 con 0x7f315c0ab7c0 2020-05-18T21:40:44.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.339+0000 7f3158ff9700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f314004fa00 0x7f3140051ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:44.355 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.339+0000 7f3158ff9700 1 -- 172.21.15.196:0/864374457 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3154064060 con 0x7f315c0ab7c0 2020-05-18T21:40:44.356 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.339+0000 7f315b7fe700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f314004fa00 0x7f3140051ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.356 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.339+0000 7f315b7fe700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f314004fa00 0x7f3140051ec0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f3144002f70 tx=0x7f3144007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.356 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.339+0000 7f315b7fe700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f314004fa00 0x7f3140051ec0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f3144002f70 tx=0x7f3144007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.356 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.339+0000 7f315b7fe700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f314004fa00 0x7f3140051ec0 secure :-1 s=READY pgs=78 cs=0 l=1 rx=0x7f3144002f70 tx=0x7f3144007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:44.356 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.347+0000 7f315bfff700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 secure :-1 s=THROTTLE_DONE pgs=33 cs=0 l=1 rx=0x7f315c060390 tx=0x7f315400c7a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.357 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.347+0000 7f3158ff9700 1 -- 172.21.15.196:0/864374457 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f315c13a9c0 con 0x7f315c0ab7c0 2020-05-18T21:40:44.463 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.463+0000 7f31620f6700 1 -- 172.21.15.196:0/864374457 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f315c13a9c0 con 0x7f315c0ab7c0 2020-05-18T21:40:44.464 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.463+0000 7f315bfff700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 secure :-1 s=THROTTLE_DONE pgs=33 cs=0 l=1 rx=0x7f315c060390 tx=0x7f315400c7a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.464 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.463+0000 7f3158ff9700 1 -- 172.21.15.196:0/864374457 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f315c13a9c0 con 0x7f315c0ab7c0 2020-05-18T21:40:44.465 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 1 2020-05-18T21:40:44.466 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.467+0000 7f31620f6700 1 -- 172.21.15.196:0/864374457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f314004fa00 msgr2=0x7f3140051ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.466 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.467+0000 7f31620f6700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f314004fa00 0x7f3140051ec0 secure :-1 s=READY pgs=78 cs=0 l=1 rx=0x7f3144002f70 tx=0x7f3144007a80).stop 2020-05-18T21:40:44.466 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.467+0000 7f31620f6700 1 -- 172.21.15.196:0/864374457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 msgr2=0x7f315c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.467 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.467+0000 7f31620f6700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 secure :-1 s=READY pgs=33 cs=0 l=1 rx=0x7f315c060390 tx=0x7f315400c7a0).stop 2020-05-18T21:40:44.467 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.467+0000 7f31620f6700 1 -- 172.21.15.196:0/864374457 shutdown_connections 2020-05-18T21:40:44.467 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.467+0000 7f31620f6700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f314004fa00 0x7f3140051ec0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.467 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.467+0000 7f31620f6700 1 --2- 172.21.15.196:0/864374457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f315c0ab7c0 0x7f315c0a2a70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.467 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.467+0000 7f31620f6700 1 --2- 172.21.15.196:0/864374457 >> v2:172.21.15.80:3300/0 conn(0x7f315c065c30 0x7f315c1379c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.468 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.467+0000 7f31620f6700 1 -- 172.21.15.196:0/864374457 >> 172.21.15.196:0/864374457 conn(0x7f315c069ac0 msgr2=0x7f315c13bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:44.468 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.467+0000 7f31620f6700 1 -- 172.21.15.196:0/864374457 shutdown_connections 2020-05-18T21:40:44.468 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.467+0000 7f31620f6700 1 -- 172.21.15.196:0/864374457 wait complete. 2020-05-18T21:40:44.476 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-05-18T21:40:44.587 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.587+0000 7fc79c876700 1 Processor -- start 2020-05-18T21:40:44.587 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.587+0000 7fc79c876700 1 -- start start 2020-05-18T21:40:44.588 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.587+0000 7fc79c876700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc79805c1b0 0x7fc7980a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:44.588 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.587+0000 7fc79c876700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79805b670 con 0x7fc79813aac0 2020-05-18T21:40:44.589 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.587+0000 7fc79c876700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7980a9760 con 0x7fc798066680 2020-05-18T21:40:44.589 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.587+0000 7fc79c876700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79813b630 con 0x7fc79805c1b0 2020-05-18T21:40:44.589 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.587+0000 7fc7977fe700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc798066680 0x7fc79805baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43690/0 (socket says 172.21.15.196:43690) 2020-05-18T21:40:44.589 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.587+0000 7fc7977fe700 1 -- 172.21.15.196:0/2979318271 learned_addr learned my addr 172.21.15.196:0/2979318271 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:44.589 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.587+0000 7fc796ffd700 1 --2- 172.21.15.196:0/2979318271 >> v2:172.21.15.80:3300/0 conn(0x7fc79805c1b0 0x7fc7980a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.590 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.587+0000 7fc7967fc700 1 -- 172.21.15.196:0/2979318271 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046798326 0 0) 0x7fc7980a9760 con 0x7fc798066680 2020-05-18T21:40:44.590 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.587+0000 7fc7967fc700 1 -- 172.21.15.196:0/2979318271 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc778003270 con 0x7fc798066680 2020-05-18T21:40:44.595 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.587+0000 7fc796ffd700 1 -- 172.21.15.196:0/2979318271 >> v1:172.21.15.196:6789/0 conn(0x7fc798066680 legacy=0x7fc79805baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.596 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.587+0000 7fc7967fc700 1 -- 172.21.15.196:0/2979318271 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1605543579 0 0) 0x7fc778003270 con 0x7fc798066680 2020-05-18T21:40:44.596 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc796ffd700 1 -- 172.21.15.196:0/2979318271 >> v1:172.21.15.80:6789/0 conn(0x7fc79813aac0 legacy=0x7fc79813af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.596 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc796ffd700 1 -- 172.21.15.196:0/2979318271 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980a9a00 con 0x7fc79805c1b0 2020-05-18T21:40:44.596 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc796ffd700 1 --2- 172.21.15.196:0/2979318271 >> v2:172.21.15.80:3300/0 conn(0x7fc79805c1b0 0x7fc7980a91e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fc790001490 tx=0x7fc79805e6a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.596 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc796ffd700 1 --2- 172.21.15.196:0/2979318271 >> v2:172.21.15.80:3300/0 conn(0x7fc79805c1b0 0x7fc7980a91e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fc790001490 tx=0x7fc79805e6a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.597 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc796ffd700 1 --2- 172.21.15.196:0/2979318271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79805c1b0 0x7fc7980a91e0 secure :-1 s=READY pgs=142 cs=0 l=1 rx=0x7fc790001490 tx=0x7fc79805e6a0).ready entity=mon.0 client_cookie=282bcc5186377090 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:44.597 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc796ffd700 1 --2- 172.21.15.196:0/2979318271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79805c1b0 0x7fc7980a91e0 secure :-1 s=THROTTLE_DONE pgs=142 cs=0 l=1 rx=0x7fc790001490 tx=0x7fc79805e6a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.597 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7967fc700 1 -- 172.21.15.196:0/2979318271 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc780001890 con 0x7fc79805c1b0 2020-05-18T21:40:44.597 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc796ffd700 1 --2- 172.21.15.196:0/2979318271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79805c1b0 0x7fc7980a91e0 secure :-1 s=THROTTLE_DONE pgs=142 cs=0 l=1 rx=0x7fc790001490 tx=0x7fc79805e6a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.598 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc796ffd700 1 --2- 172.21.15.196:0/2979318271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79805c1b0 0x7fc7980a91e0 secure :-1 s=THROTTLE_DONE pgs=142 cs=0 l=1 rx=0x7fc790001490 tx=0x7fc79805e6a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.599 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7967fc700 1 -- 172.21.15.196:0/2979318271 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788008b40 con 0x7fc79805c1b0 2020-05-18T21:40:44.599 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7967fc700 1 -- 172.21.15.196:0/2979318271 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc788008ea0 con 0x7fc79805c1b0 2020-05-18T21:40:44.599 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 -- 172.21.15.196:0/2979318271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79805c1b0 msgr2=0x7fc7980a91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.599 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 --2- 172.21.15.196:0/2979318271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79805c1b0 0x7fc7980a91e0 secure :-1 s=READY pgs=142 cs=0 l=1 rx=0x7fc790001490 tx=0x7fc79805e6a0).stop 2020-05-18T21:40:44.599 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 -- 172.21.15.196:0/2979318271 shutdown_connections 2020-05-18T21:40:44.600 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 --2- 172.21.15.196:0/2979318271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc79805c1b0 0x7fc7980a91e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.600 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 -- 172.21.15.196:0/2979318271 >> 172.21.15.196:0/2979318271 conn(0x7fc798069ac0 msgr2=0x7fc798065c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:44.601 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 -- 172.21.15.196:0/2979318271 shutdown_connections 2020-05-18T21:40:44.602 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 -- 172.21.15.196:0/2979318271 wait complete. 2020-05-18T21:40:44.602 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 Processor -- start 2020-05-18T21:40:44.602 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 -- start start 2020-05-18T21:40:44.602 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc79805c1b0 0x7fc798145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:44.603 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc798066680 0x7fc7980a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:44.603 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc79813aac0 0x7fc7980a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:44.603 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7980a4ef0 con 0x7fc79813aac0 2020-05-18T21:40:44.603 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc798145c70 con 0x7fc79805c1b0 2020-05-18T21:40:44.604 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980a2830 con 0x7fc798066680 2020-05-18T21:40:44.604 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7977fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc79805c1b0 0x7fc798145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.604 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7977fe700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc79805c1b0 0x7fc798145810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34936/0 (socket says 172.21.15.196:34936) 2020-05-18T21:40:44.605 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7977fe700 1 -- 172.21.15.196:0/1747382438 learned_addr learned my addr 172.21.15.196:0/1747382438 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:44.605 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc796ffd700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc798066680 0x7fc7980a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.605 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc797fff700 1 --2- 172.21.15.196:0/1747382438 >> v2:172.21.15.80:3300/0 conn(0x7fc79813aac0 0x7fc7980a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.606 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7977fe700 1 -- 172.21.15.196:0/1747382438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc798066680 msgr2=0x7fc7980a52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.607 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7977fe700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc798066680 0x7fc7980a52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.607 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7977fe700 1 -- 172.21.15.196:0/1747382438 >> v2:172.21.15.80:3300/0 conn(0x7fc79813aac0 msgr2=0x7fc7980a34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.607 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7977fe700 1 --2- 172.21.15.196:0/1747382438 >> v2:172.21.15.80:3300/0 conn(0x7fc79813aac0 0x7fc7980a34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.608 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7977fe700 1 -- 172.21.15.196:0/1747382438 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980a2ad0 con 0x7fc79805c1b0 2020-05-18T21:40:44.608 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc797fff700 1 --2- 172.21.15.196:0/1747382438 >> v2:172.21.15.80:3300/0 conn(0x7fc79813aac0 0x7fc7980a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request state changed! 2020-05-18T21:40:44.608 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7977fe700 1 --2- 172.21.15.196:0/1747382438 >> v2:172.21.15.196:3300/0 conn(0x7fc79805c1b0 0x7fc798145810 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc780001af0 tx=0x7fc79805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.608 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7977fe700 1 --2- 172.21.15.196:0/1747382438 >> v2:172.21.15.196:3300/0 conn(0x7fc79805c1b0 0x7fc798145810 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc780001af0 tx=0x7fc79805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.608 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7977fe700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79805c1b0 0x7fc798145810 secure :-1 s=READY pgs=192 cs=0 l=1 rx=0x7fc780001af0 tx=0x7fc79805e380).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:44.609 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc7977fe700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79805c1b0 0x7fc798145810 secure :-1 s=THROTTLE_DONE pgs=192 cs=0 l=1 rx=0x7fc780001af0 tx=0x7fc79805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.609 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc794ff9700 1 -- 172.21.15.196:0/1747382438 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc78000ca50 con 0x7fc79805c1b0 2020-05-18T21:40:44.609 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.591+0000 7fc79c876700 1 -- 172.21.15.196:0/1747382438 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7980a5740 con 0x7fc79805c1b0 2020-05-18T21:40:44.609 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc7977fe700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79805c1b0 0x7fc798145810 secure :-1 s=THROTTLE_DONE pgs=192 cs=0 l=1 rx=0x7fc780001af0 tx=0x7fc79805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.609 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc794ff9700 1 -- 172.21.15.196:0/1747382438 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78000cbb0 con 0x7fc79805c1b0 2020-05-18T21:40:44.610 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc79c876700 1 -- 172.21.15.196:0/1747382438 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc798139370 con 0x7fc79805c1b0 2020-05-18T21:40:44.610 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc7977fe700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79805c1b0 0x7fc798145810 secure :-1 s=THROTTLE_DONE pgs=192 cs=0 l=1 rx=0x7fc780001af0 tx=0x7fc79805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.610 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc794ff9700 1 -- 172.21.15.196:0/1747382438 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7800054f0 con 0x7fc79805c1b0 2020-05-18T21:40:44.610 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc7977fe700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79805c1b0 0x7fc798145810 secure :-1 s=THROTTLE_DONE pgs=192 cs=0 l=1 rx=0x7fc780001af0 tx=0x7fc79805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.610 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc79c876700 1 -- 172.21.15.196:0/1747382438 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7980a3ac0 con 0x7fc79805c1b0 2020-05-18T21:40:44.611 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc794ff9700 1 -- 172.21.15.196:0/1747382438 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fc780005650 con 0x7fc79805c1b0 2020-05-18T21:40:44.611 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc7977fe700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79805c1b0 0x7fc798145810 secure :-1 s=THROTTLE_DONE pgs=192 cs=0 l=1 rx=0x7fc780001af0 tx=0x7fc79805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.611 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc794ff9700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc77c04f8c0 0x7fc77c051d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:44.611 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc794ff9700 1 -- 172.21.15.196:0/1747382438 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc7800100c0 con 0x7fc79805c1b0 2020-05-18T21:40:44.611 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc796ffd700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc77c04f8c0 0x7fc77c051d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.612 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc796ffd700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc77c04f8c0 0x7fc77c051d80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc790001490 tx=0x7fc79805e6a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.613 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc796ffd700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc77c04f8c0 0x7fc77c051d80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc790001490 tx=0x7fc79805e6a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.613 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.595+0000 7fc796ffd700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc77c04f8c0 0x7fc77c051d80 secure :-1 s=READY pgs=79 cs=0 l=1 rx=0x7fc790001490 tx=0x7fc79805e6a0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:44.613 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.599+0000 7fc7977fe700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79805c1b0 0x7fc798145810 secure :-1 s=THROTTLE_DONE pgs=192 cs=0 l=1 rx=0x7fc780001af0 tx=0x7fc79805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.613 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.599+0000 7fc794ff9700 1 -- 172.21.15.196:0/1747382438 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7fc78003e230 con 0x7fc79805c1b0 2020-05-18T21:40:44.712 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.711+0000 7fc79c876700 1 -- 172.21.15.196:0/1747382438 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc7980a2650 con 0x7fc79805c1b0 2020-05-18T21:40:44.713 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.711+0000 7fc7977fe700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79805c1b0 0x7fc798145810 secure :-1 s=THROTTLE_DONE pgs=192 cs=0 l=1 rx=0x7fc780001af0 tx=0x7fc79805e380).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.713 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.711+0000 7fc794ff9700 1 -- 172.21.15.196:0/1747382438 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc780041850 con 0x7fc79805c1b0 2020-05-18T21:40:44.713 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 8 2020-05-18T21:40:44.715 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.715+0000 7fc79c876700 1 -- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc77c04f8c0 msgr2=0x7fc77c051d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.716 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.715+0000 7fc79c876700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc77c04f8c0 0x7fc77c051d80 secure :-1 s=READY pgs=79 cs=0 l=1 rx=0x7fc790001490 tx=0x7fc79805e6a0).stop 2020-05-18T21:40:44.716 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.715+0000 7fc79c876700 1 -- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79805c1b0 msgr2=0x7fc798145810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.717 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.715+0000 7fc79c876700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79805c1b0 0x7fc798145810 secure :-1 s=READY pgs=192 cs=0 l=1 rx=0x7fc780001af0 tx=0x7fc79805e380).stop 2020-05-18T21:40:44.717 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.715+0000 7fc79c876700 1 -- 172.21.15.196:0/1747382438 shutdown_connections 2020-05-18T21:40:44.717 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.715+0000 7fc79c876700 1 --2- 172.21.15.196:0/1747382438 >> v2:172.21.15.80:3300/0 conn(0x7fc79813aac0 0x7fc7980a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.717 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.715+0000 7fc79c876700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fc77c04f8c0 0x7fc77c051d80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.718 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.715+0000 7fc79c876700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc798066680 0x7fc7980a52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.718 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.715+0000 7fc79c876700 1 --2- 172.21.15.196:0/1747382438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc79805c1b0 0x7fc798145810 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.718 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.715+0000 7fc79c876700 1 -- 172.21.15.196:0/1747382438 >> 172.21.15.196:0/1747382438 conn(0x7fc798069ac0 msgr2=0x7fc79805ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:44.718 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.715+0000 7fc79c876700 1 -- 172.21.15.196:0/1747382438 shutdown_connections 2020-05-18T21:40:44.718 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.715+0000 7fc79c876700 1 -- 172.21.15.196:0/1747382438 wait complete. 2020-05-18T21:40:44.725 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-05-18T21:40:44.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.835+0000 7fa4f6761700 1 Processor -- start 2020-05-18T21:40:44.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.835+0000 7fa4f6761700 1 -- start start 2020-05-18T21:40:44.839 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.835+0000 7fa4f6761700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f00669f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:44.840 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.835+0000 7fa4f6761700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa4f00a3280 0x7fa4f01380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:44.840 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.835+0000 7fa4f6761700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f00a36f0 con 0x7fa4f0069e00 2020-05-18T21:40:44.840 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.835+0000 7fa4f6761700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f0066e50 con 0x7fa4f00a3280 2020-05-18T21:40:44.840 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.835+0000 7fa4f6761700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00a3920 con 0x7fa4f00a7200 2020-05-18T21:40:44.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4ef7fe700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f00669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4f4cfe700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa4f00a3280 0x7fa4f01380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4ef7fe700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f00669f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.196:38778/0 (socket says 172.21.15.196:38778) 2020-05-18T21:40:44.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4ef7fe700 1 -- 172.21.15.196:0/2857077087 learned_addr learned my addr 172.21.15.196:0/2857077087 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:44.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4eeffd700 1 -- 172.21.15.196:0/2857077087 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3928096918 0 0) 0x7fa4f00a36f0 con 0x7fa4f0069e00 2020-05-18T21:40:44.841 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4eeffd700 1 -- 172.21.15.196:0/2857077087 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4dc003270 con 0x7fa4f0069e00 2020-05-18T21:40:44.842 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4ef7fe700 1 -- 172.21.15.196:0/2857077087 >> v2:172.21.15.80:3300/0 conn(0x7fa4f00a3280 msgr2=0x7fa4f01380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:44.842 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4ef7fe700 1 --2- 172.21.15.196:0/2857077087 >> v2:172.21.15.80:3300/0 conn(0x7fa4f00a3280 0x7fa4f01380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:44.842 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4ef7fe700 1 -- 172.21.15.196:0/2857077087 >> v1:172.21.15.80:6789/0 conn(0x7fa4f0069e00 legacy=0x7fa4f00a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.842 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4ef7fe700 1 -- 172.21.15.196:0/2857077087 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f00a6d60 con 0x7fa4f00a7200 2020-05-18T21:40:44.842 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4ef7fe700 1 --2- 172.21.15.196:0/2857077087 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f00669f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fa4e4002f70 tx=0x7fa4e4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.843 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4ef7fe700 1 --2- 172.21.15.196:0/2857077087 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f00669f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fa4e4002f70 tx=0x7fa4e4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.843 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4ef7fe700 1 --2- 172.21.15.196:0/2857077087 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f00669f0 secure :-1 s=READY pgs=34 cs=0 l=1 rx=0x7fa4e4002f70 tx=0x7fa4e4007a80).ready entity=mon.2 client_cookie=1d0055050d9ac37e server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:44.843 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4ef7fe700 1 --2- 172.21.15.196:0/2857077087 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f00669f0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7fa4e4002f70 tx=0x7fa4e4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.843 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4eeffd700 1 -- 172.21.15.196:0/2857077087 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e4011600 con 0x7fa4f00a7200 2020-05-18T21:40:44.843 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4ef7fe700 1 --2- 172.21.15.196:0/2857077087 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f00669f0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7fa4e4002f70 tx=0x7fa4e4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.844 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4ef7fe700 1 --2- 172.21.15.196:0/2857077087 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f00669f0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7fa4e4002f70 tx=0x7fa4e4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.844 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4eeffd700 1 -- 172.21.15.196:0/2857077087 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e4011760 con 0x7fa4f00a7200 2020-05-18T21:40:44.844 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4f6761700 1 -- 172.21.15.196:0/2857077087 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 msgr2=0x7fa4f00669f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.844 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4eeffd700 1 -- 172.21.15.196:0/2857077087 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e4011b80 con 0x7fa4f00a7200 2020-05-18T21:40:44.844 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4f6761700 1 --2- 172.21.15.196:0/2857077087 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f00669f0 secure :-1 s=READY pgs=34 cs=0 l=1 rx=0x7fa4e4002f70 tx=0x7fa4e4007a80).stop 2020-05-18T21:40:44.849 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4f6761700 1 -- 172.21.15.196:0/2857077087 shutdown_connections 2020-05-18T21:40:44.849 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4f6761700 1 --2- 172.21.15.196:0/2857077087 >> v2:172.21.15.80:3300/0 conn(0x7fa4f00a3280 0x7fa4f01380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:44.849 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4f6761700 1 --2- 172.21.15.196:0/2857077087 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f00669f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.850 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4f6761700 1 -- 172.21.15.196:0/2857077087 >> 172.21.15.196:0/2857077087 conn(0x7fa4f005a7c0 msgr2=0x7fa4f005abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:44.850 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4f6761700 1 -- 172.21.15.196:0/2857077087 shutdown_connections 2020-05-18T21:40:44.850 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4f6761700 1 -- 172.21.15.196:0/2857077087 wait complete. 2020-05-18T21:40:44.850 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4f6761700 1 Processor -- start 2020-05-18T21:40:44.851 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.839+0000 7fa4f6761700 1 -- start start 2020-05-18T21:40:44.851 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4f6761700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa4f0069e00 0x7fa4f00aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:44.851 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4f6761700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f013b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:44.851 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4f6761700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f01363f0 con 0x7fa4f00a3280 2020-05-18T21:40:44.851 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4f6761700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f0137370 con 0x7fa4f0069e00 2020-05-18T21:40:44.852 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4f6761700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f0136650 con 0x7fa4f00a7200 2020-05-18T21:40:44.852 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4ef7fe700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa4f00a3280 0x7fa4f0138080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43712/0 (socket says 172.21.15.196:43712) 2020-05-18T21:40:44.852 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4ef7fe700 1 -- 172.21.15.196:0/698039662 learned_addr learned my addr 172.21.15.196:0/698039662 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:44.852 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4effff700 1 --2- 172.21.15.196:0/698039662 >> v2:172.21.15.80:3300/0 conn(0x7fa4f0069e00 0x7fa4f00aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.852 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4f4cfe700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f013b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.853 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4ed7fa700 1 -- 172.21.15.196:0/698039662 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189062622 0 0) 0x7fa4f01363f0 con 0x7fa4f00a3280 2020-05-18T21:40:44.853 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4ed7fa700 1 -- 172.21.15.196:0/698039662 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4d0002fe0 con 0x7fa4f00a3280 2020-05-18T21:40:44.853 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4effff700 1 -- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 msgr2=0x7fa4f013b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.865 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4effff700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f013b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.865 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4effff700 1 -- 172.21.15.196:0/698039662 >> v1:172.21.15.196:6789/0 conn(0x7fa4f00a3280 legacy=0x7fa4f0138080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.865 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4effff700 1 -- 172.21.15.196:0/698039662 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f01368f0 con 0x7fa4f0069e00 2020-05-18T21:40:44.865 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4effff700 1 --2- 172.21.15.196:0/698039662 >> v2:172.21.15.80:3300/0 conn(0x7fa4f0069e00 0x7fa4f00aaee0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa4e0001160 tx=0x7fa4f005e600).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.866 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4effff700 1 --2- 172.21.15.196:0/698039662 >> v2:172.21.15.80:3300/0 conn(0x7fa4f0069e00 0x7fa4f00aaee0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa4e0001160 tx=0x7fa4f005e600).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.866 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4effff700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4f0069e00 0x7fa4f00aaee0 secure :-1 s=READY pgs=144 cs=0 l=1 rx=0x7fa4e0001160 tx=0x7fa4f005e600).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:44.866 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4effff700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4f0069e00 0x7fa4f00aaee0 secure :-1 s=THROTTLE_DONE pgs=144 cs=0 l=1 rx=0x7fa4e0001160 tx=0x7fa4f005e600).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.867 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4ed7fa700 1 -- 172.21.15.196:0/698039662 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e0006c50 con 0x7fa4f0069e00 2020-05-18T21:40:44.867 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4f6761700 1 -- 172.21.15.196:0/698039662 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f0138790 con 0x7fa4f0069e00 2020-05-18T21:40:44.867 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4f6761700 1 -- 172.21.15.196:0/698039662 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f013bca0 con 0x7fa4f0069e00 2020-05-18T21:40:44.867 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4effff700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4f0069e00 0x7fa4f00aaee0 secure :-1 s=THROTTLE_DONE pgs=144 cs=0 l=1 rx=0x7fa4e0001160 tx=0x7fa4f005e600).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.867 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4effff700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4f0069e00 0x7fa4f00aaee0 secure :-1 s=THROTTLE_DONE pgs=144 cs=0 l=1 rx=0x7fa4e0001160 tx=0x7fa4f005e600).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.868 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4ed7fa700 1 -- 172.21.15.196:0/698039662 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e0006db0 con 0x7fa4f0069e00 2020-05-18T21:40:44.868 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4ed7fa700 1 -- 172.21.15.196:0/698039662 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e0009460 con 0x7fa4f0069e00 2020-05-18T21:40:44.868 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4effff700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4f0069e00 0x7fa4f00aaee0 secure :-1 s=THROTTLE_DONE pgs=144 cs=0 l=1 rx=0x7fa4e0001160 tx=0x7fa4f005e600).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.868 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.843+0000 7fa4f6761700 1 -- 172.21.15.196:0/698039662 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f00569e0 con 0x7fa4f0069e00 2020-05-18T21:40:44.868 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.847+0000 7fa4ed7fa700 1 -- 172.21.15.196:0/698039662 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fa4e00096d0 con 0x7fa4f0069e00 2020-05-18T21:40:44.868 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.847+0000 7fa4effff700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4f0069e00 0x7fa4f00aaee0 secure :-1 s=THROTTLE_DONE pgs=144 cs=0 l=1 rx=0x7fa4e0001160 tx=0x7fa4f005e600).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.869 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.847+0000 7fa4ed7fa700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa4d004fd80 0x7fa4d0052240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:44.869 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.847+0000 7fa4ed7fa700 1 -- 172.21.15.196:0/698039662 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa4e0063d40 con 0x7fa4f0069e00 2020-05-18T21:40:44.869 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.847+0000 7fa4ef7fe700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa4d004fd80 0x7fa4d0052240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:44.869 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.847+0000 7fa4ef7fe700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa4d004fd80 0x7fa4d0052240 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa4e4000bc0 tx=0x7fa4e4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.870 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.847+0000 7fa4ef7fe700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa4d004fd80 0x7fa4d0052240 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa4e4000bc0 tx=0x7fa4e4007a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.870 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.847+0000 7fa4ef7fe700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa4d004fd80 0x7fa4d0052240 secure :-1 s=READY pgs=80 cs=0 l=1 rx=0x7fa4e4000bc0 tx=0x7fa4e4007a80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:44.871 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.855+0000 7fa4effff700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4f0069e00 0x7fa4f00aaee0 secure :-1 s=THROTTLE_DONE pgs=144 cs=0 l=1 rx=0x7fa4e0001160 tx=0x7fa4f005e600).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.872 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.855+0000 7fa4ed7fa700 1 -- 172.21.15.196:0/698039662 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7fa4e003e0d0 con 0x7fa4f0069e00 2020-05-18T21:40:44.972 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.971+0000 7fa4f6761700 1 -- 172.21.15.196:0/698039662 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4f0068f40 con 0x7fa4f0069e00 2020-05-18T21:40:44.973 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.971+0000 7fa4effff700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4f0069e00 0x7fa4f00aaee0 secure :-1 s=THROTTLE_DONE pgs=144 cs=0 l=1 rx=0x7fa4e0001160 tx=0x7fa4f005e600).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:44.973 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.971+0000 7fa4ed7fa700 1 -- 172.21.15.196:0/698039662 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa4e00416f0 con 0x7fa4f0069e00 2020-05-18T21:40:44.973 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 8 2020-05-18T21:40:44.974 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.975+0000 7fa4f6761700 1 -- 172.21.15.196:0/698039662 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa4d004fd80 msgr2=0x7fa4d0052240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.974 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.975+0000 7fa4f6761700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa4d004fd80 0x7fa4d0052240 secure :-1 s=READY pgs=80 cs=0 l=1 rx=0x7fa4e4000bc0 tx=0x7fa4e4007a80).stop 2020-05-18T21:40:44.975 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.975+0000 7fa4f6761700 1 -- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4f0069e00 msgr2=0x7fa4f00aaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:44.975 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.975+0000 7fa4f6761700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4f0069e00 0x7fa4f00aaee0 secure :-1 s=READY pgs=144 cs=0 l=1 rx=0x7fa4e0001160 tx=0x7fa4f005e600).stop 2020-05-18T21:40:44.975 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.975+0000 7fa4f6761700 1 -- 172.21.15.196:0/698039662 shutdown_connections 2020-05-18T21:40:44.975 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.975+0000 7fa4f6761700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fa4d004fd80 0x7fa4d0052240 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.975 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.975+0000 7fa4f6761700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4f00a7200 0x7fa4f013b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.976 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.975+0000 7fa4f6761700 1 --2- 172.21.15.196:0/698039662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4f0069e00 0x7fa4f00aaee0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:44.976 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.975+0000 7fa4f6761700 1 -- 172.21.15.196:0/698039662 >> 172.21.15.196:0/698039662 conn(0x7fa4f005a7c0 msgr2=0x7fa4f005eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:44.976 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.975+0000 7fa4f6761700 1 -- 172.21.15.196:0/698039662 shutdown_connections 2020-05-18T21:40:44.976 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:44.975+0000 7fa4f6761700 1 -- 172.21.15.196:0/698039662 wait complete. 2020-05-18T21:40:44.985 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-05-18T21:40:45.088 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.087+0000 7fe3cb3d1700 1 Processor -- start 2020-05-18T21:40:45.089 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.087+0000 7fe3cb3d1700 1 -- start start 2020-05-18T21:40:45.091 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe3c40a96d0 0x7fe3c4060340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:45.091 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe3c405aed0 0x7fe3c405b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:45.091 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c40a8c50 con 0x7fe3c40a7e10 2020-05-18T21:40:45.091 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c40a6ba0 con 0x7fe3c40a96d0 2020-05-18T21:40:45.092 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c40609d0 con 0x7fe3c405aed0 2020-05-18T21:40:45.092 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cabd0700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe3c405aed0 0x7fe3c405b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:45.092 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3c9bce700 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe3c40a7e10 0x7fe3c405a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:43722/0 (socket says 172.21.15.196:43722) 2020-05-18T21:40:45.092 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3c9bce700 1 -- 172.21.15.196:0/656055812 learned_addr learned my addr 172.21.15.196:0/656055812 (peer_addr_for_me v1:172.21.15.196:0/0) 2020-05-18T21:40:45.092 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3ca3cf700 1 --2- 172.21.15.196:0/656055812 >> v2:172.21.15.80:3300/0 conn(0x7fe3c40a96d0 0x7fe3c4060340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:45.093 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3c93cd700 1 -- 172.21.15.196:0/656055812 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213071137 0 0) 0x7fe3c40a8c50 con 0x7fe3c40a7e10 2020-05-18T21:40:45.093 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3c93cd700 1 -- 172.21.15.196:0/656055812 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3ac003270 con 0x7fe3c40a7e10 2020-05-18T21:40:45.093 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cabd0700 1 -- 172.21.15.196:0/656055812 >> v2:172.21.15.80:3300/0 conn(0x7fe3c40a96d0 msgr2=0x7fe3c4060340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:45.093 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cabd0700 1 --2- 172.21.15.196:0/656055812 >> v2:172.21.15.80:3300/0 conn(0x7fe3c40a96d0 0x7fe3c4060340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:45.093 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cabd0700 1 -- 172.21.15.196:0/656055812 >> v1:172.21.15.196:6789/0 conn(0x7fe3c40a7e10 legacy=0x7fe3c405a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:45.094 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3c93cd700 1 -- 172.21.15.196:0/656055812 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3966620388 0 0) 0x7fe3ac003270 con 0x7fe3c40a7e10 2020-05-18T21:40:45.094 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cabd0700 1 -- 172.21.15.196:0/656055812 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c40607a0 con 0x7fe3c405aed0 2020-05-18T21:40:45.094 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cabd0700 1 --2- 172.21.15.196:0/656055812 >> v2:172.21.15.196:3300/0 conn(0x7fe3c405aed0 0x7fe3c405b340 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fe3b40018b0 tx=0x7fe3bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.094 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cabd0700 1 --2- 172.21.15.196:0/656055812 >> v2:172.21.15.196:3300/0 conn(0x7fe3c405aed0 0x7fe3c405b340 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fe3b40018b0 tx=0x7fe3bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.094 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cabd0700 1 --2- 172.21.15.196:0/656055812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c405aed0 0x7fe3c405b340 secure :-1 s=READY pgs=195 cs=0 l=1 rx=0x7fe3b40018b0 tx=0x7fe3bc007e00).ready entity=mon.1 client_cookie=b82af74f38fbf595 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:45.095 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cabd0700 1 --2- 172.21.15.196:0/656055812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c405aed0 0x7fe3c405b340 secure :-1 s=THROTTLE_DONE pgs=195 cs=0 l=1 rx=0x7fe3b40018b0 tx=0x7fe3bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.095 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3c93cd700 1 -- 172.21.15.196:0/656055812 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3b4001710 con 0x7fe3c405aed0 2020-05-18T21:40:45.095 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cabd0700 1 --2- 172.21.15.196:0/656055812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c405aed0 0x7fe3c405b340 secure :-1 s=THROTTLE_DONE pgs=195 cs=0 l=1 rx=0x7fe3b40018b0 tx=0x7fe3bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.095 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3c93cd700 1 -- 172.21.15.196:0/656055812 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3bc010b00 con 0x7fe3c405aed0 2020-05-18T21:40:45.095 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cabd0700 1 --2- 172.21.15.196:0/656055812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c405aed0 0x7fe3c405b340 secure :-1 s=THROTTLE_DONE pgs=195 cs=0 l=1 rx=0x7fe3b40018b0 tx=0x7fe3bc007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.096 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/656055812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c405aed0 msgr2=0x7fe3c405b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:45.096 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3c93cd700 1 -- 172.21.15.196:0/656055812 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3bc010ea0 con 0x7fe3c405aed0 2020-05-18T21:40:45.096 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 --2- 172.21.15.196:0/656055812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c405aed0 0x7fe3c405b340 secure :-1 s=READY pgs=195 cs=0 l=1 rx=0x7fe3b40018b0 tx=0x7fe3bc007e00).stop 2020-05-18T21:40:45.099 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/656055812 shutdown_connections 2020-05-18T21:40:45.099 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 --2- 172.21.15.196:0/656055812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c405aed0 0x7fe3c405b340 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:45.099 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 --2- 172.21.15.196:0/656055812 >> v2:172.21.15.80:3300/0 conn(0x7fe3c40a96d0 0x7fe3c4060340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:45.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/656055812 >> 172.21.15.196:0/656055812 conn(0x7fe3c405c460 msgr2=0x7fe3c4062850 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:45.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/656055812 shutdown_connections 2020-05-18T21:40:45.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/656055812 wait complete. 2020-05-18T21:40:45.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 Processor -- start 2020-05-18T21:40:45.100 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.091+0000 7fe3cb3d1700 1 -- start start 2020-05-18T21:40:45.101 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3cb3d1700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe3c405aed0 0x7fe3c4138a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:45.101 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3cb3d1700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe3c40a7e10 0x7fe3c40aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:45.101 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3cb3d1700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c40a1d50 con 0x7fe3c40a96d0 2020-05-18T21:40:45.101 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3cb3d1700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c40a1b40 con 0x7fe3c405aed0 2020-05-18T21:40:45.101 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3cb3d1700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c40a1ff0 con 0x7fe3c40a7e10 2020-05-18T21:40:45.102 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe3c40a7e10 0x7fe3c40aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:45.102 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe3c40a7e10 0x7fe3c40aa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34966/0 (socket says 172.21.15.196:34966) 2020-05-18T21:40:45.102 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 -- 172.21.15.196:0/31504065 learned_addr learned my addr 172.21.15.196:0/31504065 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:45.102 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3ca3cf700 1 --2- 172.21.15.196:0/31504065 >> v2:172.21.15.80:3300/0 conn(0x7fe3c405aed0 0x7fe3c4138a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:45.102 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 -- 172.21.15.196:0/31504065 >> v2:172.21.15.80:3300/0 conn(0x7fe3c405aed0 msgr2=0x7fe3c4138a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:45.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 --2- 172.21.15.196:0/31504065 >> v2:172.21.15.80:3300/0 conn(0x7fe3c405aed0 0x7fe3c4138a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:45.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 -- 172.21.15.196:0/31504065 >> v1:172.21.15.80:6789/0 conn(0x7fe3c40a96d0 legacy=0x7fe3c40ac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:45.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 -- 172.21.15.196:0/31504065 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c40acdc0 con 0x7fe3c40a7e10 2020-05-18T21:40:45.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 --2- 172.21.15.196:0/31504065 >> v2:172.21.15.196:3300/0 conn(0x7fe3c40a7e10 0x7fe3c40aa500 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe3b4001970 tx=0x7fe3b4010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 --2- 172.21.15.196:0/31504065 >> v2:172.21.15.196:3300/0 conn(0x7fe3c40a7e10 0x7fe3c40aa500 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe3b4001970 tx=0x7fe3b4010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.103 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c40a7e10 0x7fe3c40aa500 secure :-1 s=READY pgs=196 cs=0 l=1 rx=0x7fe3b4001970 tx=0x7fe3b4010cf0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:45.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c40a7e10 0x7fe3c40aa500 secure :-1 s=THROTTLE_DONE pgs=196 cs=0 l=1 rx=0x7fe3b4001970 tx=0x7fe3b4010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3b37fe700 1 -- 172.21.15.196:0/31504065 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3b4012a30 con 0x7fe3c40a7e10 2020-05-18T21:40:45.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c40a7e10 0x7fe3c40aa500 secure :-1 s=THROTTLE_DONE pgs=196 cs=0 l=1 rx=0x7fe3b4001970 tx=0x7fe3b4010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c40a7e10 0x7fe3c40aa500 secure :-1 s=THROTTLE_DONE pgs=196 cs=0 l=1 rx=0x7fe3b4001970 tx=0x7fe3b4010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.104 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3b37fe700 1 -- 172.21.15.196:0/31504065 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b4012b90 con 0x7fe3c40a7e10 2020-05-18T21:40:45.105 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/31504065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c4139030 con 0x7fe3c40a7e10 2020-05-18T21:40:45.105 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3b37fe700 1 -- 172.21.15.196:0/31504065 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3b400c5b0 con 0x7fe3c40a7e10 2020-05-18T21:40:45.105 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/31504065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c40aa960 con 0x7fe3c40a7e10 2020-05-18T21:40:45.105 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c40a7e10 0x7fe3c40aa500 secure :-1 s=THROTTLE_DONE pgs=196 cs=0 l=1 rx=0x7fe3b4001970 tx=0x7fe3b4010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.105 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3b37fe700 1 -- 172.21.15.196:0/31504065 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7fe3b400c820 con 0x7fe3c40a7e10 2020-05-18T21:40:45.106 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3c9bce700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c40a7e10 0x7fe3c40aa500 secure :-1 s=THROTTLE_DONE pgs=196 cs=0 l=1 rx=0x7fe3b4001970 tx=0x7fe3b4010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.106 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/31504065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c40569e0 con 0x7fe3c40a7e10 2020-05-18T21:40:45.106 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3b37fe700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe3a404fa10 0x7fe3a4051ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:45.106 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3b37fe700 1 -- 172.21.15.196:0/31504065 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe3b4063e20 con 0x7fe3c40a7e10 2020-05-18T21:40:45.106 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3ca3cf700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe3a404fa10 0x7fe3a4051ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:45.106 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3ca3cf700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe3a404fa10 0x7fe3a4051ed0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe3c0002f70 tx=0x7fe3c000d750).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.107 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3ca3cf700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe3a404fa10 0x7fe3a4051ed0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe3c0002f70 tx=0x7fe3c000d750).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.107 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.095+0000 7fe3ca3cf700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe3a404fa10 0x7fe3a4051ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rx=0x7fe3c0002f70 tx=0x7fe3c000d750).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:45.107 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.103+0000 7fe3c9bce700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c40a7e10 0x7fe3c40aa500 secure :-1 s=THROTTLE_DONE pgs=196 cs=0 l=1 rx=0x7fe3b4001970 tx=0x7fe3b4010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.108 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.103+0000 7fe3b37fe700 1 -- 172.21.15.196:0/31504065 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7fe3b403e150 con 0x7fe3c40a7e10 2020-05-18T21:40:45.217 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/31504065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3c40a2130 con 0x7fe3c40a7e10 2020-05-18T21:40:45.219 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3c9bce700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c40a7e10 0x7fe3c40aa500 secure :-1 s=THROTTLE_DONE pgs=196 cs=0 l=1 rx=0x7fe3b4001970 tx=0x7fe3b4010cf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.219 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3b37fe700 1 -- 172.21.15.196:0/31504065 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe3b4041770 con 0x7fe3c40a7e10 2020-05-18T21:40:45.219 INFO:teuthology.orchestra.run.smithi196.stdout:pg_num: 8 2020-05-18T21:40:45.220 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe3a404fa10 msgr2=0x7fe3a4051ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:45.220 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3cb3d1700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe3a404fa10 0x7fe3a4051ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rx=0x7fe3c0002f70 tx=0x7fe3c000d750).stop 2020-05-18T21:40:45.220 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c40a7e10 msgr2=0x7fe3c40aa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:45.221 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3cb3d1700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c40a7e10 0x7fe3c40aa500 secure :-1 s=READY pgs=196 cs=0 l=1 rx=0x7fe3b4001970 tx=0x7fe3b4010cf0).stop 2020-05-18T21:40:45.221 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/31504065 shutdown_connections 2020-05-18T21:40:45.221 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3cb3d1700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7fe3a404fa10 0x7fe3a4051ed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:45.221 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3cb3d1700 1 --2- 172.21.15.196:0/31504065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3c40a7e10 0x7fe3c40aa500 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:45.222 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3cb3d1700 1 --2- 172.21.15.196:0/31504065 >> v2:172.21.15.80:3300/0 conn(0x7fe3c405aed0 0x7fe3c4138a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:45.222 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/31504065 >> 172.21.15.196:0/31504065 conn(0x7fe3c405c460 msgr2=0x7fe3c4062770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:45.222 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/31504065 shutdown_connections 2020-05-18T21:40:45.222 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.219+0000 7fe3cb3d1700 1 -- 172.21.15.196:0/31504065 wait complete. 2020-05-18T21:40:45.231 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-05-18T21:40:45.231 INFO:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-05-18T21:40:45.329 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 Processor -- start 2020-05-18T21:40:45.330 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 -- start start 2020-05-18T21:40:45.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f85e80a4080 0x7f85e8140130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:45.332 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85e8140590 0x7f85e8146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-05-18T21:40:45.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e809e0b0 con 0x7f85e80a7220 2020-05-18T21:40:45.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e80a3580 con 0x7f85e80a4080 2020-05-18T21:40:45.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80a20f0 con 0x7f85e8140590 2020-05-18T21:40:45.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e7fff700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f85e80a4080 0x7f85e8140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:45.333 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e7fff700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f85e80a4080 0x7f85e8140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.196:39198/0 (socket says 172.21.15.196:39198) 2020-05-18T21:40:45.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e7fff700 1 -- 172.21.15.196:0/1243549777 learned_addr learned my addr 172.21.15.196:0/1243549777 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:45.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85ed6b8700 1 --2- 172.21.15.196:0/1243549777 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85e8140590 0x7f85e8146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:45.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e7fff700 1 -- 172.21.15.196:0/1243549777 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85e8140590 msgr2=0x7f85e8146fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-05-18T21:40:45.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e7fff700 1 --2- 172.21.15.196:0/1243549777 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85e8140590 0x7f85e8146fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:45.334 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e7fff700 1 -- 172.21.15.196:0/1243549777 >> v1:172.21.15.80:6789/0 conn(0x7f85e80a7220 legacy=0x7f85e80a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:45.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e7fff700 1 -- 172.21.15.196:0/1243549777 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e80a6a60 con 0x7f85e80a4080 2020-05-18T21:40:45.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85ed6b8700 1 --2- 172.21.15.196:0/1243549777 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85e8140590 0x7f85e8146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).handle_auth_reply_more state changed! 2020-05-18T21:40:45.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e7fff700 1 --2- 172.21.15.196:0/1243549777 >> v2:172.21.15.80:3300/0 conn(0x7f85e80a4080 0x7f85e8140130 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e7fff700 1 --2- 172.21.15.196:0/1243549777 >> v2:172.21.15.80:3300/0 conn(0x7f85e80a4080 0x7f85e8140130 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e7fff700 1 --2- 172.21.15.196:0/1243549777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80a4080 0x7f85e8140130 secure :-1 s=READY pgs=147 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).ready entity=mon.0 client_cookie=293a4bfabfbbd37f server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:45.335 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e7fff700 1 --2- 172.21.15.196:0/1243549777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80a4080 0x7f85e8140130 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e77fe700 1 -- 172.21.15.196:0/1243549777 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85e000a180 con 0x7f85e80a4080 2020-05-18T21:40:45.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e7fff700 1 --2- 172.21.15.196:0/1243549777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80a4080 0x7f85e8140130 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e7fff700 1 --2- 172.21.15.196:0/1243549777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80a4080 0x7f85e8140130 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e77fe700 1 -- 172.21.15.196:0/1243549777 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e000a2e0 con 0x7f85e80a4080 2020-05-18T21:40:45.336 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 -- 172.21.15.196:0/1243549777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80a4080 msgr2=0x7f85e8140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:45.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85e77fe700 1 -- 172.21.15.196:0/1243549777 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85e000a700 con 0x7f85e80a4080 2020-05-18T21:40:45.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 --2- 172.21.15.196:0/1243549777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80a4080 0x7f85e8140130 secure :-1 s=READY pgs=147 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).stop 2020-05-18T21:40:45.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 -- 172.21.15.196:0/1243549777 shutdown_connections 2020-05-18T21:40:45.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 --2- 172.21.15.196:0/1243549777 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85e8140590 0x7f85e8146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop 2020-05-18T21:40:45.337 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 --2- 172.21.15.196:0/1243549777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80a4080 0x7f85e8140130 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:45.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 -- 172.21.15.196:0/1243549777 >> 172.21.15.196:0/1243549777 conn(0x7f85e805a7c0 msgr2=0x7f85e805abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:45.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 -- 172.21.15.196:0/1243549777 shutdown_connections 2020-05-18T21:40:45.338 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.331+0000 7f85edeb9700 1 -- 172.21.15.196:0/1243549777 wait complete. 2020-05-18T21:40:45.340 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85edeb9700 1 Processor -- start 2020-05-18T21:40:45.341 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85edeb9700 1 -- start start 2020-05-18T21:40:45.341 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85edeb9700 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f85e80a4080 0x7f85e813d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:45.341 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85edeb9700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f85e80a7220 0x7f85e814a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:45.341 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85edeb9700 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85e8140590 0x7f85e809f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:45.341 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85edeb9700 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e8099b10 con 0x7f85e80a4080 2020-05-18T21:40:45.342 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85edeb9700 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e81496a0 con 0x7f85e80a7220 2020-05-18T21:40:45.342 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85edeb9700 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80a6530 con 0x7f85e8140590 2020-05-18T21:40:45.342 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f85e80a7220 0x7f85e814a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:45.342 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f85e80a7220 0x7f85e814a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34980/0 (socket says 172.21.15.196:34980) 2020-05-18T21:40:45.342 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 -- 172.21.15.196:0/820370522 learned_addr learned my addr 172.21.15.196:0/820370522 (peer_addr_for_me v2:172.21.15.196:0/0) 2020-05-18T21:40:45.343 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85ed6b8700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85e8140590 0x7f85e809f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:45.343 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85eceb7700 1 --2- 172.21.15.196:0/820370522 >> v2:172.21.15.80:3300/0 conn(0x7f85e80a4080 0x7f85e813d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:45.343 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 -- 172.21.15.196:0/820370522 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85e8140590 msgr2=0x7f85e809f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:45.343 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85e8140590 0x7f85e809f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:45.343 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 -- 172.21.15.196:0/820370522 >> v2:172.21.15.80:3300/0 conn(0x7f85e80a4080 msgr2=0x7f85e813d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:45.344 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 --2- 172.21.15.196:0/820370522 >> v2:172.21.15.80:3300/0 conn(0x7f85e80a4080 0x7f85e813d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:45.344 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 -- 172.21.15.196:0/820370522 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e809f6b0 con 0x7f85e80a7220 2020-05-18T21:40:45.344 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 --2- 172.21.15.196:0/820370522 >> v2:172.21.15.196:3300/0 conn(0x7f85e80a7220 0x7f85e814a270 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.344 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 --2- 172.21.15.196:0/820370522 >> v2:172.21.15.196:3300/0 conn(0x7f85e80a7220 0x7f85e814a270 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.344 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e80a7220 0x7f85e814a270 secure :-1 s=READY pgs=197 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:45.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e80a7220 0x7f85e814a270 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e80a7220 0x7f85e814a270 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e5ffb700 1 -- 172.21.15.196:0/820370522 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85e0005a80 con 0x7f85e80a7220 2020-05-18T21:40:45.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e80a7220 0x7f85e814a270 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e5ffb700 1 -- 172.21.15.196:0/820370522 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e0005be0 con 0x7f85e80a7220 2020-05-18T21:40:45.345 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85edeb9700 1 -- 172.21.15.196:0/820370522 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e81498f0 con 0x7f85e80a7220 2020-05-18T21:40:45.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e5ffb700 1 -- 172.21.15.196:0/820370522 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85e0007170 con 0x7f85e80a7220 2020-05-18T21:40:45.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85edeb9700 1 -- 172.21.15.196:0/820370522 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e8099e90 con 0x7f85e80a7220 2020-05-18T21:40:45.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e80a7220 0x7f85e814a270 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85edeb9700 1 -- 172.21.15.196:0/820370522 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e80569e0 con 0x7f85e80a7220 2020-05-18T21:40:45.346 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e5ffb700 1 -- 172.21.15.196:0/820370522 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 4) v1 ==== 61621+0+0 (secure 0 0 0) 0x7f85e00073a0 con 0x7f85e80a7220 2020-05-18T21:40:45.347 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.335+0000 7f85e7fff700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e80a7220 0x7f85e814a270 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.347 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.339+0000 7f85e5ffb700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f85cc04f9f0 0x7f85cc051eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-05-18T21:40:45.347 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.339+0000 7f85e5ffb700 1 -- 172.21.15.196:0/820370522 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f85e0064720 con 0x7f85e80a7220 2020-05-18T21:40:45.347 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.339+0000 7f85eceb7700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f85cc04f9f0 0x7f85cc051eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-05-18T21:40:45.347 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.339+0000 7f85eceb7700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f85cc04f9f0 0x7f85cc051eb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f85e80602b0 tx=0x7f85d0003ee0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.339+0000 7f85eceb7700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f85cc04f9f0 0x7f85cc051eb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f85e80602b0 tx=0x7f85d0003ee0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.339+0000 7f85eceb7700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f85cc04f9f0 0x7f85cc051eb0 secure :-1 s=READY pgs=82 cs=0 l=1 rx=0x7f85e80602b0 tx=0x7f85d0003ee0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-05-18T21:40:45.348 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.343+0000 7f85e7fff700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e80a7220 0x7f85e814a270 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.349 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.343+0000 7f85e5ffb700 1 -- 172.21.15.196:0/820370522 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+129441 (secure 0 0 0) 0x7f85e003ebd0 con 0x7f85e80a7220 2020-05-18T21:40:45.488 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.487+0000 7f85edeb9700 1 -- 172.21.15.196:0/820370522 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85e809a450 con 0x7f85e80a7220 2020-05-18T21:40:45.489 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.487+0000 7f85e7fff700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e80a7220 0x7f85e814a270 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-05-18T21:40:45.489 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.487+0000 7f85e5ffb700 1 -- 172.21.15.196:0/820370522 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2131 (secure 0 0 0) 0x7f85e00421f0 con 0x7f85e80a7220 2020-05-18T21:40:45.489 INFO:teuthology.orchestra.run.smithi196.stdout: 2020-05-18T21:40:45.489 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.196:6835/993135284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":993135284},{"type":"v1","addr":"172.21.15.196:6835","nonce":993135284}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0,"epoch":7}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-05-18T21:40:32.208271+0000","modified":"2020-05-18T21:40:36.226347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4386":{"gid":4386,"name":"b","rank":0,"incarnation":5,"state":"up:active","state_seq":2,"addr":"172.21.15.80:6833/386820289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":386820289},{"type":"v1","addr":"172.21.15.80:6833","nonce":386820289}]},"join_fscid":-1,"export_targets":[],"features":4540138292837744639,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-05-18T21:40:45.491 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.491+0000 7f85edeb9700 1 -- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f85cc04f9f0 msgr2=0x7f85cc051eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:45.491 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.491+0000 7f85edeb9700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f85cc04f9f0 0x7f85cc051eb0 secure :-1 s=READY pgs=82 cs=0 l=1 rx=0x7f85e80602b0 tx=0x7f85d0003ee0).stop 2020-05-18T21:40:45.491 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.491+0000 7f85edeb9700 1 -- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e80a7220 msgr2=0x7f85e814a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-05-18T21:40:45.491 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.491+0000 7f85edeb9700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e80a7220 0x7f85e814a270 secure :-1 s=READY pgs=197 cs=0 l=1 rx=0x7f85e00032f0 tx=0x7f85e0007e00).stop 2020-05-18T21:40:45.492 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.491+0000 7f85edeb9700 1 -- 172.21.15.196:0/820370522 shutdown_connections 2020-05-18T21:40:45.492 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.491+0000 7f85edeb9700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:6832/12657,v1:172.21.15.196:6833/12657] conn(0x7f85cc04f9f0 0x7f85cc051eb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:45.492 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.491+0000 7f85edeb9700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85e8140590 0x7f85e809f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:45.492 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.491+0000 7f85edeb9700 1 --2- 172.21.15.196:0/820370522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85e80a7220 0x7f85e814a270 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:45.492 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.491+0000 7f85edeb9700 1 --2- 172.21.15.196:0/820370522 >> v2:172.21.15.80:3300/0 conn(0x7f85e80a4080 0x7f85e813d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rx=0 tx=0).stop 2020-05-18T21:40:45.493 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.491+0000 7f85edeb9700 1 -- 172.21.15.196:0/820370522 >> 172.21.15.196:0/820370522 conn(0x7f85e805a7c0 msgr2=0x7f85e805eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-05-18T21:40:45.493 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.491+0000 7f85edeb9700 1 -- 172.21.15.196:0/820370522 shutdown_connections 2020-05-18T21:40:45.493 INFO:teuthology.orchestra.run.smithi196.stderr:2020-05-18T21:40:45.491+0000 7f85edeb9700 1 -- 172.21.15.196:0/820370522 wait complete. 2020-05-18T21:40:45.493 INFO:teuthology.orchestra.run.smithi196.stderr:dumped fsmap epoch 7 2020-05-18T21:40:45.703 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-05-18T21:40:32.208271+0000', 'modified': '2020-05-18T21:40:36.226347+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4386}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4386': {'gid': 4386, 'name': 'b', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.80:6833/386820289', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6832', 'nonce': 386820289}, {'type': 'v1', 'addr': '172.21.15.80:6833', 'nonce': 386820289}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292837744639, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-05-18T21:40:45.703 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-05-18T21:40:45.704 INFO:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok status 2020-05-18T21:40:45.809 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2020-05-18T21:40:45.810 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "d048673a-3d91-4cc6-882d-e9cc70e7cd6c", 2020-05-18T21:40:45.810 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2020-05-18T21:40:45.810 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 4386, 2020-05-18T21:40:45.810 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2020-05-18T21:40:45.810 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2020-05-18T21:40:45.811 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 10.58019857, 2020-05-18T21:40:45.811 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 6, 2020-05-18T21:40:45.811 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 22, 2020-05-18T21:40:45.811 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 22, 2020-05-18T21:40:45.811 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 11.232717575000001 2020-05-18T21:40:45.812 INFO:teuthology.orchestra.run.smithi080.stdout:} 2020-05-18T21:40:45.818 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "d048673a-3d91-4cc6-882d-e9cc70e7cd6c", "whoami": 0, "id": 4386, "want_state": "up:active", "state": "up:active", "rank_uptime": 10.58019857, "mdsmap_epoch": 6, "osdmap_epoch": 22, "osdmap_epoch_barrier": 22, "uptime": 11.232717575000001 } 2020-05-18T21:40:45.818 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-05-18T21:40:45.819 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2020-05-18T21:40:45.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:45.819 INFO:teuthology.orchestra.run.smithi196:> ip addr 2020-05-18T21:40:45.828 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-05-18T21:40:45.828 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-05-18T21:40:45.829 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2020-05-18T21:40:45.829 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2020-05-18T21:40:45.829 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2020-05-18T21:40:45.829 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2020-05-18T21:40:45.829 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-05-18T21:40:45.830 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2020-05-18T21:40:45.830 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-05-18T21:40:45.830 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2020-05-18T21:40:45.830 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-05-18T21:40:45.830 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2020-05-18T21:40:45.831 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-05-18T21:40:45.831 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2020-05-18T21:40:45.831 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2020-05-18T21:40:45.831 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42799sec preferred_lft 42799sec 2020-05-18T21:40:45.831 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2020-05-18T21:40:45.832 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2020-05-18T21:40:45.832 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-05-18T21:40:45.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:45.833 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'ip link add name ceph-brx type bridge' 2020-05-18T21:40:45.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:45.944 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'ip link set ceph-brx up' 2020-05-18T21:40:45.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:45.959 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx' 2020-05-18T21:40:45.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:45.972 INFO:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-05-18T21:40:45.982 INFO:teuthology.orchestra.run.smithi196.stdout:1 2020-05-18T21:40:45.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:45.983 INFO:teuthology.orchestra.run.smithi196:> route 2020-05-18T21:40:46.050 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2020-05-18T21:40:46.051 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-05-18T21:40:46.051 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-05-18T21:40:46.051 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2020-05-18T21:40:46.051 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-05-18T21:40:46.052 INFO:teuthology.orchestra.run.smithi196.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2020-05-18T21:40:46.052 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-05-18T21:40:46.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.053 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT' 2020-05-18T21:40:46.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.128 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT' 2020-05-18T21:40:46.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.146 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE' 2020-05-18T21:40:46.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.261 INFO:teuthology.orchestra.run.smithi196:> ip netns list 2020-05-18T21:40:46.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.269 INFO:teuthology.orchestra.run.smithi196:> ip netns list-id 2020-05-18T21:40:46.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.276 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0' 2020-05-18T21:40:46.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.339 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0' 2020-05-18T21:40:46.355 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2020-05-18T21:40:46.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.356 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0' 2020-05-18T21:40:46.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.393 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0' 2020-05-18T21:40:46.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.408 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up' 2020-05-18T21:40:46.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.467 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up' 2020-05-18T21:40:46.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.484 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254' 2020-05-18T21:40:46.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.498 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'ip link set brx.0 up' 2020-05-18T21:40:46.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-05-18T21:40:46.559 INFO:teuthology.orchestra.run.smithi196:> sudo bash -c 'ip link set dev brx.0 master ceph-brx' 2020-05-18T21:40:46.573 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-05-18T21:40:46.574 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi196.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-05-18T21:40:46.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-05-18T21:40:46.574 INFO:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.0) 2020-05-18T21:40:46.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-05-18T21:40:46.581 INFO:teuthology.orchestra.run.smithi196:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-05-18T21:40:46.635 INFO:teuthology.orchestra.run.smithi196.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-05-18T21:40:46.636 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-05-18T21:40:46.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-05-18T21:40:46.637 INFO:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2020-05-18T21:40:46.642 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-05-18T21:40:46.643 INFO:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-05-18T21:40:46.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-05-18T21:40:46.646 INFO:teuthology.orchestra.run.smithi196:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-05-18T21:40:46.660 INFO:teuthology.orchestra.run.smithi196.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-05-18T21:40:46.661 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-05-18T21:40:46.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-05-18T21:40:46.662 INFO:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2020-05-18T21:40:46.738 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi196.stderr:2020-05-18T21:40:46.738+0000 7fcf7f4ab080 -1 init, newargv = 0x5598516d0740 newargc=9ceph-fuse[15099]: starting ceph client 2020-05-18T21:40:46.738 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi196.stderr: 2020-05-18T21:40:46.747 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi196.stderr:ceph-fuse[15099]: starting fuse 2020-05-18T21:40:47.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-05-18T21:40:47.770 INFO:teuthology.orchestra.run.smithi196:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-05-18T21:40:47.785 INFO:teuthology.orchestra.run.smithi196.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2020-05-18T21:40:47.787 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-05-18T21:40:47.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-05-18T21:40:47.787 INFO:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2020-05-18T21:40:47.793 INFO:teuthology.orchestra.run.smithi196.stdout:50 2020-05-18T21:40:47.793 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [50] 2020-05-18T21:40:47.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-05-18T21:40:47.794 INFO:teuthology.orchestra.run.smithi196:> sudo python3 -c ' 2020-05-18T21:40:47.794 INFO:teuthology.orchestra.run.smithi196:> import glob 2020-05-18T21:40:47.795 INFO:teuthology.orchestra.run.smithi196:> import re 2020-05-18T21:40:47.795 INFO:teuthology.orchestra.run.smithi196:> import os 2020-05-18T21:40:47.795 INFO:teuthology.orchestra.run.smithi196:> import subprocess 2020-05-18T21:40:47.796 INFO:teuthology.orchestra.run.smithi196:> 2020-05-18T21:40:47.796 INFO:teuthology.orchestra.run.smithi196:> def find_socket(client_name): 2020-05-18T21:40:47.797 INFO:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-05-18T21:40:47.797 INFO:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2020-05-18T21:40:47.797 INFO:teuthology.orchestra.run.smithi196:> 2020-05-18T21:40:47.797 INFO:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2020-05-18T21:40:47.798 INFO:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2020-05-18T21:40:47.798 INFO:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2020-05-18T21:40:47.799 INFO:teuthology.orchestra.run.smithi196:> return files[0] 2020-05-18T21:40:47.799 INFO:teuthology.orchestra.run.smithi196:> 2020-05-18T21:40:47.800 INFO:teuthology.orchestra.run.smithi196:> for f in files: 2020-05-18T21:40:47.800 INFO:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-05-18T21:40:47.801 INFO:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2020-05-18T21:40:47.801 INFO:teuthology.orchestra.run.smithi196:> return f 2020-05-18T21:40:47.802 INFO:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-05-18T21:40:47.802 INFO:teuthology.orchestra.run.smithi196:> 2020-05-18T21:40:47.803 INFO:teuthology.orchestra.run.smithi196:> print(find_socket("client.0")) 2020-05-18T21:40:47.804 INFO:teuthology.orchestra.run.smithi196:> ' 2020-05-18T21:40:47.848 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.0.15099.asok 2020-05-18T21:40:47.853 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15099.asok 2020-05-18T21:40:47.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-05-18T21:40:47.854 INFO:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15099.asok status 2020-05-18T21:40:47.987 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2020-05-18T21:40:47.988 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2020-05-18T21:40:47.988 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "543315c934420269aa12ef2f9dec2c9eadb4fa6f", 2020-05-18T21:40:47.989 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 16.0.0-1640-g543315c9344 (543315c934420269aa12ef2f9dec2c9eadb4fa6f) pacific (dev)", 2020-05-18T21:40:47.989 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "0", 2020-05-18T21:40:47.989 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2020-05-18T21:40:47.990 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-05-18T21:40:47.990 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "15099", 2020-05-18T21:40:47.990 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2020-05-18T21:40:47.990 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2020-05-18T21:40:47.991 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2020-05-18T21:40:47.991 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2020-05-18T21:40:47.991 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 4666, 2020-05-18T21:40:47.991 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2020-05-18T21:40:47.992 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2020-05-18T21:40:47.992 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2020-05-18T21:40:47.992 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 4666 2020-05-18T21:40:47.992 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2020-05-18T21:40:47.993 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2020-05-18T21:40:47.993 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2020-05-18T21:40:47.993 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2020-05-18T21:40:47.993 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 3097440708 2020-05-18T21:40:47.993 INFO:teuthology.orchestra.run.smithi196.stdout: } 2020-05-18T21:40:47.994 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2020-05-18T21:40:47.994 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2020-05-18T21:40:47.994 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2020-05-18T21:40:47.994 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2020-05-18T21:40:47.994 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 3097440708 2020-05-18T21:40:47.995 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2020-05-18T21:40:47.995 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.4666 192.168.0.1:0/3097440708", 2020-05-18T21:40:47.995 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/3097440708", 2020-05-18T21:40:47.995 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2020-05-18T21:40:47.995 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 6, 2020-05-18T21:40:47.996 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 22, 2020-05-18T21:40:47.996 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2020-05-18T21:40:47.996 INFO:teuthology.orchestra.run.smithi196.stdout: "blacklisted": false 2020-05-18T21:40:47.996 INFO:teuthology.orchestra.run.smithi196.stdout:} 2020-05-18T21:40:47.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-05-18T21:40:47.999 INFO:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-05-18T21:40:47.999 INFO:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-05-18T21:40:48.006 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2020-05-18T21:40:48.007 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-05-18T21:40:48.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-05-18T21:40:48.008 INFO:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-05-18T21:40:48.027 INFO:teuthology.run_tasks:Running task check-counter... 2020-05-18T21:40:48.042 INFO:teuthology.run_tasks:Running task workunit... 2020-05-18T21:40:48.075 INFO:tasks.workunit:Pulling workunits from ref 543315c934420269aa12ef2f9dec2c9eadb4fa6f 2020-05-18T21:40:48.076 INFO:tasks.workunit:Making a separate scratch dir for every client... 2020-05-18T21:40:48.077 INFO:tasks.workunit:timeout=3h 2020-05-18T21:40:48.077 INFO:tasks.workunit:cleanup=True 2020-05-18T21:40:48.078 INFO:teuthology.orchestra.run.smithi196:> stat -- /home/ubuntu/cephtest/mnt.0 2020-05-18T21:40:48.089 INFO:teuthology.orchestra.run.smithi196.stdout: File: /home/ubuntu/cephtest/mnt.0 2020-05-18T21:40:48.089 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2020-05-18T21:40:48.089 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 32h/50d Inode: 1 Links: 2 2020-05-18T21:40:48.090 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-18T21:40:48.090 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2020-05-18T21:40:48.090 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2020-05-18 21:40:35.231799404 +0000 2020-05-18T21:40:48.090 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2020-05-18 21:40:48.027995484 +0000 2020-05-18T21:40:48.091 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2020-05-18T21:40:48.091 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2020-05-18T21:40:48.091 INFO:teuthology.orchestra.run.smithi196:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2020-05-18T21:40:48.162 INFO:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 543315c934420269aa12ef2f9dec2c9eadb4fa6f 2020-05-18T21:40:48.227 INFO:tasks.workunit.client.0.smithi196.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2020-05-18T21:41:11.707 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:41:11.711 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:41:30.893 INFO:tasks.workunit.client.0.smithi196.stderr:Note: switching to '543315c934420269aa12ef2f9dec2c9eadb4fa6f'. 2020-05-18T21:41:30.894 INFO:tasks.workunit.client.0.smithi196.stderr: 2020-05-18T21:41:30.894 INFO:tasks.workunit.client.0.smithi196.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2020-05-18T21:41:30.895 INFO:tasks.workunit.client.0.smithi196.stderr:changes and commit them, and you can discard any commits you make in this 2020-05-18T21:41:30.895 INFO:tasks.workunit.client.0.smithi196.stderr:state without impacting any branches by switching back to a branch. 2020-05-18T21:41:30.895 INFO:tasks.workunit.client.0.smithi196.stderr: 2020-05-18T21:41:30.895 INFO:tasks.workunit.client.0.smithi196.stderr:If you want to create a new branch to retain commits you create, you may 2020-05-18T21:41:30.895 INFO:tasks.workunit.client.0.smithi196.stderr:do so (now or later) by using -c with the switch command. Example: 2020-05-18T21:41:30.896 INFO:tasks.workunit.client.0.smithi196.stderr: 2020-05-18T21:41:30.896 INFO:tasks.workunit.client.0.smithi196.stderr: git switch -c 2020-05-18T21:41:30.896 INFO:tasks.workunit.client.0.smithi196.stderr: 2020-05-18T21:41:30.896 INFO:tasks.workunit.client.0.smithi196.stderr:Or undo this operation with: 2020-05-18T21:41:30.896 INFO:tasks.workunit.client.0.smithi196.stderr: 2020-05-18T21:41:30.897 INFO:tasks.workunit.client.0.smithi196.stderr: git switch - 2020-05-18T21:41:30.897 INFO:tasks.workunit.client.0.smithi196.stderr: 2020-05-18T21:41:30.897 INFO:tasks.workunit.client.0.smithi196.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2020-05-18T21:41:30.897 INFO:tasks.workunit.client.0.smithi196.stderr: 2020-05-18T21:41:30.897 INFO:tasks.workunit.client.0.smithi196.stderr:HEAD is now at 543315c9344 Merge PR #34144 into wip-pdonnell-testing-20200516.030215 2020-05-18T21:41:30.899 INFO:teuthology.orchestra.run.smithi196:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2020-05-18T21:41:30.943 INFO:tasks.workunit.client.0.smithi196.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2020-05-18T21:41:30.946 INFO:tasks.workunit.client.0.smithi196.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2020-05-18T21:41:30.946 INFO:tasks.workunit.client.0.smithi196.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2020-05-18T21:41:31.854 INFO:tasks.workunit.client.0.smithi196.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2020-05-18T21:41:31.937 INFO:tasks.workunit.client.0.smithi196.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2020-05-18T21:41:31.990 INFO:tasks.workunit.client.0.smithi196.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2020-05-18T21:41:31.992 INFO:tasks.workunit.client.0.smithi196.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2020-05-18T21:41:31.993 INFO:tasks.workunit.client.0.smithi196.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2020-05-18T21:41:32.046 INFO:tasks.workunit.client.0.smithi196.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2020-05-18T21:41:32.065 DEBUG:teuthology.orchestra.remote:smithi196:/home/ubuntu/cephtest/workunits.list.client.0 is 5KB 2020-05-18T21:41:32.080 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2020-05-18T21:41:32.080 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2020-05-18T21:41:32.081 INFO:teuthology.orchestra.run.smithi196:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=543315c934420269aa12ef2f9dec2c9eadb4fa6f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2020-05-18T21:41:32.151 INFO:tasks.workunit.client.0.smithi196.stderr:--2020-05-18 21:41:32-- http://download.ceph.com/qa/ffsb.tar.bz2 2020-05-18T21:41:32.153 INFO:tasks.workunit.client.0.smithi196.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2020-05-18T21:41:32.240 INFO:tasks.workunit.client.0.smithi196.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2020-05-18T21:41:32.329 INFO:tasks.workunit.client.0.smithi196.stderr:HTTP request sent, awaiting response... 200 OK 2020-05-18T21:41:32.329 INFO:tasks.workunit.client.0.smithi196.stderr:Length: 128931 (126K) [application/octet-stream] 2020-05-18T21:41:32.338 INFO:tasks.workunit.client.0.smithi196.stderr:Saving to: 'ffsb.tar.bz2' 2020-05-18T21:41:32.341 INFO:tasks.workunit.client.0.smithi196.stderr: 2020-05-18T21:41:32.504 INFO:tasks.workunit.client.0.smithi196.stderr: 0K .......... .......... .......... .......... .......... 39% 295K 0s 2020-05-18T21:41:32.596 INFO:tasks.workunit.client.0.smithi196.stderr: 50K .......... .......... .......... .......... .......... 79% 552K 0s 2020-05-18T21:41:32.597 INFO:tasks.workunit.client.0.smithi196.stderr: 100K .......... .......... ..... 100% 11.0M=0.3s 2020-05-18T21:41:32.598 INFO:tasks.workunit.client.0.smithi196.stderr: 2020-05-18T21:41:32.598 INFO:tasks.workunit.client.0.smithi196.stderr:2020-05-18 21:41:32 (480 KB/s) - 'ffsb.tar.bz2' saved [128931/128931] 2020-05-18T21:41:32.599 INFO:tasks.workunit.client.0.smithi196.stderr: 2020-05-18T21:41:32.647 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ 2020-05-18T21:41:32.650 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ffsb_fs.c 2020-05-18T21:41:32.654 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/INSTALL 2020-05-18T21:41:32.659 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/fileops.c 2020-05-18T21:41:32.663 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/missing 2020-05-18T21:41:32.667 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/config.h.in 2020-05-18T21:41:32.671 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/config.guess 2020-05-18T21:41:32.675 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/fh.h 2020-05-18T21:41:32.678 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/COPYING 2020-05-18T21:41:32.682 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/util.h 2020-05-18T21:41:32.685 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/main.c 2020-05-18T21:41:32.687 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ChangeLog 2020-05-18T21:41:32.690 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/rand.c 2020-05-18T21:41:32.692 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ffsb_fs.h 2020-05-18T21:41:32.695 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/install-sh 2020-05-18T21:41:32.698 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/rwlock.h 2020-05-18T21:41:32.701 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/parser.h 2020-05-18T21:41:32.704 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/filelist.h 2020-05-18T21:41:32.708 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/LICENSE 2020-05-18T21:41:32.711 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/Makefile.in 2020-05-18T21:41:32.720 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/filelist.c 2020-05-18T21:41:32.720 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/configure 2020-05-18T21:41:32.727 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/README 2020-05-18T21:41:32.731 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/cirlist.c 2020-05-18T21:41:32.735 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/metaops.c 2020-05-18T21:41:32.738 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/registration-description 2020-05-18T21:41:32.742 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/list.c 2020-05-18T21:41:32.745 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/list.h 2020-05-18T21:41:32.749 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/rand.h 2020-05-18T21:41:32.752 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/parser.c 2020-05-18T21:41:32.755 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ffsb_tg.c 2020-05-18T21:41:32.758 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/cirlist.h 2020-05-18T21:41:32.768 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/AUTHORS 2020-05-18T21:41:32.768 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/aclocal.m4 2020-05-18T21:41:32.768 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/configure.in 2020-05-18T21:41:32.771 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/examples/ 2020-05-18T21:41:32.773 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/examples/profile_everything 2020-05-18T21:41:32.777 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/depcomp 2020-05-18T21:41:32.782 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ffsb_stats.c 2020-05-18T21:41:32.786 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/util.c 2020-05-18T21:41:32.790 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/Makefile.am 2020-05-18T21:41:32.794 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ffsb_thread.c 2020-05-18T21:41:32.797 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/rbt.c 2020-05-18T21:41:32.801 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/rbt.h 2020-05-18T21:41:32.805 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ffsb_tg.h 2020-05-18T21:41:32.808 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/stamp-h.in 2020-05-18T21:41:32.811 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/config.sub 2020-05-18T21:41:32.816 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ffsb_thread.h 2020-05-18T21:41:32.819 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/fileops.h 2020-05-18T21:41:32.823 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ffsb_op.h 2020-05-18T21:41:32.826 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/public-description 2020-05-18T21:41:32.829 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/mkinstalldirs 2020-05-18T21:41:32.833 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ffsb.h 2020-05-18T21:41:32.836 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/metaops.h 2020-05-18T21:41:32.839 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ffsb_op.c 2020-05-18T21:41:32.842 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/fh.c 2020-05-18T21:41:32.847 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/NEWS 2020-05-18T21:41:32.848 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/rwlock.c 2020-05-18T21:41:32.851 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ffsb_stats.h 2020-05-18T21:41:32.854 INFO:tasks.workunit.client.0.smithi196.stdout:ffsb-6.0-rc2/ffsb_fc.c 2020-05-18T21:41:32.886 INFO:tasks.workunit.client.0.smithi196.stdout:patching file parser.c 2020-05-18T21:41:33.277 INFO:tasks.workunit.client.0.smithi196.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2020-05-18T21:41:34.297 INFO:tasks.workunit.client.0.smithi196.stdout:checking whether build environment is sane... yes 2020-05-18T21:41:34.335 INFO:tasks.workunit.client.0.smithi196.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2020-05-18T21:41:34.336 INFO:tasks.workunit.client.0.smithi196.stdout:checking for gawk... gawk 2020-05-18T21:41:34.386 INFO:tasks.workunit.client.0.smithi196.stdout:checking whether make sets $(MAKE)... yes 2020-05-18T21:41:34.497 INFO:tasks.workunit.client.0.smithi196.stdout:checking build system type... x86_64-unknown-linux-gnu 2020-05-18T21:41:34.498 INFO:tasks.workunit.client.0.smithi196.stdout:checking host system type... x86_64-unknown-linux-gnu 2020-05-18T21:41:34.541 INFO:tasks.workunit.client.0.smithi196.stdout:checking for style of include used by make... GNU 2020-05-18T21:41:34.553 INFO:tasks.workunit.client.0.smithi196.stdout:checking for gcc... gcc 2020-05-18T21:41:34.731 INFO:tasks.workunit.client.0.smithi196.stdout:checking for C compiler default output file name... a.out 2020-05-18T21:41:34.738 INFO:tasks.workunit.client.0.smithi196.stdout:checking whether the C compiler works... yes 2020-05-18T21:41:34.745 INFO:tasks.workunit.client.0.smithi196.stdout:checking whether we are cross compiling... no 2020-05-18T21:41:34.851 INFO:tasks.workunit.client.0.smithi196.stdout:checking for suffix of executables... 2020-05-18T21:41:34.942 INFO:tasks.workunit.client.0.smithi196.stdout:checking for suffix of object files... o 2020-05-18T21:41:35.038 INFO:tasks.workunit.client.0.smithi196.stdout:checking whether we are using the GNU C compiler... yes 2020-05-18T21:41:35.129 INFO:tasks.workunit.client.0.smithi196.stdout:checking whether gcc accepts -g... yes 2020-05-18T21:41:35.282 INFO:tasks.workunit.client.0.smithi196.stdout:checking for gcc option to accept ISO C89... none needed 2020-05-18T21:41:35.586 INFO:tasks.workunit.client.0.smithi196.stdout:checking dependency style of gcc... gcc3 2020-05-18T21:41:35.738 INFO:tasks.workunit.client.0.smithi196.stdout:checking how to run the C preprocessor... gcc -E 2020-05-18T21:41:35.860 INFO:tasks.workunit.client.0.smithi196.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2020-05-18T21:41:35.864 INFO:tasks.workunit.client.0.smithi196.stdout:checking for egrep... /usr/bin/grep -E 2020-05-18T21:41:35.890 INFO:tasks.workunit.client.0.smithi196.stdout:checking for AIX... no 2020-05-18T21:41:35.933 INFO:tasks.workunit.client.0.smithi196.stdout:checking for gcc... (cached) gcc 2020-05-18T21:41:35.963 INFO:tasks.workunit.client.0.smithi196.stdout:checking whether we are using the GNU C compiler... (cached) yes 2020-05-18T21:41:35.966 INFO:tasks.workunit.client.0.smithi196.stdout:checking whether gcc accepts -g... (cached) yes 2020-05-18T21:41:35.967 INFO:tasks.workunit.client.0.smithi196.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2020-05-18T21:41:35.969 INFO:tasks.workunit.client.0.smithi196.stdout:checking dependency style of gcc... (cached) gcc3 2020-05-18T21:41:36.112 INFO:tasks.workunit.client.0.smithi196.stdout:checking for main in -lm... yes 2020-05-18T21:41:36.264 INFO:tasks.workunit.client.0.smithi196.stdout:checking for main in -lpthread... yes 2020-05-18T21:41:36.657 INFO:tasks.workunit.client.0.smithi196.stdout:checking for ANSI C header files... yes 2020-05-18T21:41:36.773 INFO:tasks.workunit.client.0.smithi196.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2020-05-18T21:41:36.888 INFO:tasks.workunit.client.0.smithi196.stdout:checking for sys/types.h... yes 2020-05-18T21:41:37.019 INFO:tasks.workunit.client.0.smithi196.stdout:checking for sys/stat.h... yes 2020-05-18T21:41:37.150 INFO:tasks.workunit.client.0.smithi196.stdout:checking for stdlib.h... yes 2020-05-18T21:41:37.278 INFO:tasks.workunit.client.0.smithi196.stdout:checking for string.h... yes 2020-05-18T21:41:37.407 INFO:tasks.workunit.client.0.smithi196.stdout:checking for memory.h... yes 2020-05-18T21:41:37.543 INFO:tasks.workunit.client.0.smithi196.stdout:checking for strings.h... yes 2020-05-18T21:41:37.678 INFO:tasks.workunit.client.0.smithi196.stdout:checking for inttypes.h... yes 2020-05-18T21:41:37.818 INFO:tasks.workunit.client.0.smithi196.stdout:checking for stdint.h... yes 2020-05-18T21:41:37.956 INFO:tasks.workunit.client.0.smithi196.stdout:checking for unistd.h... yes 2020-05-18T21:41:38.117 INFO:tasks.workunit.client.0.smithi196.stdout:checking pthread.h usability... yes 2020-05-18T21:41:38.174 INFO:tasks.workunit.client.0.smithi196.stdout:checking pthread.h presence... yes 2020-05-18T21:41:38.175 INFO:tasks.workunit.client.0.smithi196.stdout:checking for pthread.h... yes 2020-05-18T21:41:38.339 INFO:tasks.workunit.client.0.smithi196.stdout:checking fcntl.h usability... yes 2020-05-18T21:41:38.392 INFO:tasks.workunit.client.0.smithi196.stdout:checking fcntl.h presence... yes 2020-05-18T21:41:38.394 INFO:tasks.workunit.client.0.smithi196.stdout:checking for fcntl.h... yes 2020-05-18T21:41:38.543 INFO:tasks.workunit.client.0.smithi196.stdout:checking limits.h usability... yes 2020-05-18T21:41:38.603 INFO:tasks.workunit.client.0.smithi196.stdout:checking limits.h presence... yes 2020-05-18T21:41:38.604 INFO:tasks.workunit.client.0.smithi196.stdout:checking for limits.h... yes 2020-05-18T21:41:38.648 INFO:tasks.workunit.client.0.smithi196.stdout:checking for stdint.h... (cached) yes 2020-05-18T21:41:38.774 INFO:tasks.workunit.client.0.smithi196.stdout:checking sys/time.h usability... yes 2020-05-18T21:41:38.829 INFO:tasks.workunit.client.0.smithi196.stdout:checking sys/time.h presence... yes 2020-05-18T21:41:38.831 INFO:tasks.workunit.client.0.smithi196.stdout:checking for sys/time.h... yes 2020-05-18T21:41:38.899 INFO:tasks.workunit.client.0.smithi196.stdout:checking for unistd.h... (cached) yes 2020-05-18T21:41:39.010 INFO:tasks.workunit.client.0.smithi196.stdout:checking sys/vfs.h usability... yes 2020-05-18T21:41:39.065 INFO:tasks.workunit.client.0.smithi196.stdout:checking sys/vfs.h presence... yes 2020-05-18T21:41:39.067 INFO:tasks.workunit.client.0.smithi196.stdout:checking for sys/vfs.h... yes 2020-05-18T21:41:39.203 INFO:tasks.workunit.client.0.smithi196.stdout:checking sys/limits.h usability... no 2020-05-18T21:41:39.266 INFO:tasks.workunit.client.0.smithi196.stdout:checking sys/limits.h presence... no 2020-05-18T21:41:39.268 INFO:tasks.workunit.client.0.smithi196.stdout:checking for sys/limits.h... no 2020-05-18T21:41:39.346 INFO:tasks.workunit.client.0.smithi196.stdout:checking for an ANSI C-conforming const... yes 2020-05-18T21:41:39.440 INFO:tasks.workunit.client.0.smithi196.stdout:checking whether time.h and sys/time.h may both be included... yes 2020-05-18T21:41:39.448 INFO:tasks.workunit.client.0.smithi196.stdout:checking for function prototypes... yes 2020-05-18T21:41:39.559 INFO:tasks.workunit.client.0.smithi196.stdout:checking whether setvbuf arguments are reversed... no 2020-05-18T21:41:39.724 INFO:tasks.workunit.client.0.smithi196.stdout:checking for system... yes 2020-05-18T21:41:39.891 INFO:tasks.workunit.client.0.smithi196.stdout:checking for gettimeofday... yes 2020-05-18T21:41:40.068 INFO:tasks.workunit.client.0.smithi196.stdout:checking for mkdir... yes 2020-05-18T21:41:40.233 INFO:tasks.workunit.client.0.smithi196.stdout:checking for strerror... yes 2020-05-18T21:41:40.398 INFO:tasks.workunit.client.0.smithi196.stdout:checking for open64... yes 2020-05-18T21:41:40.565 INFO:tasks.workunit.client.0.smithi196.stdout:checking for stat64... yes 2020-05-18T21:41:40.724 INFO:tasks.workunit.client.0.smithi196.stdout:checking for fseeko64... yes 2020-05-18T21:41:40.893 INFO:tasks.workunit.client.0.smithi196.stdout:checking for lrand48_r... yes 2020-05-18T21:41:41.054 INFO:tasks.workunit.client.0.smithi196.stdout:checking for srand48_r... yes 2020-05-18T21:41:41.137 INFO:tasks.workunit.client.0.smithi196.stdout:configure: creating ./config.status 2020-05-18T21:41:41.412 INFO:tasks.workunit.client.0.smithi196.stdout:config.status: creating Makefile 2020-05-18T21:41:41.452 INFO:tasks.workunit.client.0.smithi196.stdout:config.status: creating config.h 2020-05-18T21:41:41.504 INFO:tasks.workunit.client.0.smithi196.stdout:config.status: executing depfiles commands 2020-05-18T21:41:41.670 INFO:tasks.workunit.client.0.smithi196.stdout:config.status: executing default commands 2020-05-18T21:41:41.756 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:41:41.759 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:41:41.878 INFO:tasks.workunit.client.0.smithi196.stdout:make all-am 2020-05-18T21:41:41.921 INFO:tasks.workunit.client.0.smithi196.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb-6.0-rc2' 2020-05-18T21:41:41.922 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2020-05-18T21:41:42.022 INFO:tasks.workunit.client.0.smithi196.stderr:fileops.c: In function 'ffsb_readfile': 2020-05-18T21:41:42.022 INFO:tasks.workunit.client.0.smithi196.stderr:fileops.c:138:35: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t' {aka 'long unsigned int'} [-Wformat=] 2020-05-18T21:41:42.022 INFO:tasks.workunit.client.0.smithi196.stderr: 138 | printf("Error: read size %llu bytes too big " 2020-05-18T21:41:42.023 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~^ 2020-05-18T21:41:42.023 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:42.023 INFO:tasks.workunit.client.0.smithi196.stderr: | long long unsigned int 2020-05-18T21:41:42.023 INFO:tasks.workunit.client.0.smithi196.stderr: | %lu 2020-05-18T21:41:42.023 INFO:tasks.workunit.client.0.smithi196.stderr:...... 2020-05-18T21:41:42.023 INFO:tasks.workunit.client.0.smithi196.stderr: 141 | " aborting\n\n", read_size, 2020-05-18T21:41:42.024 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~~~~~~~ 2020-05-18T21:41:42.024 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:42.024 INFO:tasks.workunit.client.0.smithi196.stderr: | uint64_t {aka long unsigned int} 2020-05-18T21:41:42.024 INFO:tasks.workunit.client.0.smithi196.stderr:fileops.c:138:14: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 5 has type 'uint64_t' {aka 'long unsigned int'} [-Wformat=] 2020-05-18T21:41:42.024 INFO:tasks.workunit.client.0.smithi196.stderr: 138 | printf("Error: read size %llu bytes too big " 2020-05-18T21:41:42.025 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:42.025 INFO:tasks.workunit.client.0.smithi196.stderr:...... 2020-05-18T21:41:42.025 INFO:tasks.workunit.client.0.smithi196.stderr: 143 | filesize); 2020-05-18T21:41:42.025 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~~~~~~ 2020-05-18T21:41:42.025 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:42.025 INFO:tasks.workunit.client.0.smithi196.stderr: | uint64_t {aka long unsigned int} 2020-05-18T21:41:42.026 INFO:tasks.workunit.client.0.smithi196.stderr:fileops.c:140:29: note: format string is defined here 2020-05-18T21:41:42.026 INFO:tasks.workunit.client.0.smithi196.stderr: 140 | " for file of size %llu bytes\n" 2020-05-18T21:41:42.026 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~^ 2020-05-18T21:41:42.026 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:42.026 INFO:tasks.workunit.client.0.smithi196.stderr: | long long unsigned int 2020-05-18T21:41:42.027 INFO:tasks.workunit.client.0.smithi196.stderr: | %lu 2020-05-18T21:41:42.027 INFO:tasks.workunit.client.0.smithi196.stderr:fileops.c:144:14: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t' {aka 'long unsigned int'} [-Wformat=] 2020-05-18T21:41:42.034 INFO:tasks.workunit.client.0.smithi196.stderr: 144 | printf("minimum file size must be at least " 2020-05-18T21:41:42.034 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:42.034 INFO:tasks.workunit.client.0.smithi196.stderr: 145 | " %llu bytes\n", minfilesize); 2020-05-18T21:41:42.034 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~~~~~~~~~ 2020-05-18T21:41:42.035 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:42.035 INFO:tasks.workunit.client.0.smithi196.stderr: | uint64_t {aka long unsigned int} 2020-05-18T21:41:42.035 INFO:tasks.workunit.client.0.smithi196.stderr:fileops.c:145:12: note: format string is defined here 2020-05-18T21:41:42.035 INFO:tasks.workunit.client.0.smithi196.stderr: 145 | " %llu bytes\n", minfilesize); 2020-05-18T21:41:42.035 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~^ 2020-05-18T21:41:42.036 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:42.036 INFO:tasks.workunit.client.0.smithi196.stderr: | long long unsigned int 2020-05-18T21:41:42.036 INFO:tasks.workunit.client.0.smithi196.stderr: | %lu 2020-05-18T21:41:42.036 INFO:tasks.workunit.client.0.smithi196.stderr:fileops.c: In function 'ffsb_stat': 2020-05-18T21:41:42.037 INFO:tasks.workunit.client.0.smithi196.stderr:fileops.c:542:2: warning: implicit declaration of function 'fhstat'; did you mean 'fstat'? [-Wimplicit-function-declaration] 2020-05-18T21:41:42.037 INFO:tasks.workunit.client.0.smithi196.stderr: 542 | fhstat(curfile->name, ft, fs); 2020-05-18T21:41:42.037 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~ 2020-05-18T21:41:42.037 INFO:tasks.workunit.client.0.smithi196.stderr: | fstat 2020-05-18T21:41:42.180 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2020-05-18T21:41:42.198 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2020-05-18T21:41:42.362 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2020-05-18T21:41:42.377 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2020-05-18T21:41:42.584 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/main.Tpo .deps/main.Po 2020-05-18T21:41:42.601 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2020-05-18T21:41:42.719 INFO:tasks.workunit.client.0.smithi196.stderr:fh.c: In function 'fhwrite': 2020-05-18T21:41:42.720 INFO:tasks.workunit.client.0.smithi196.stderr:fh.c:180:18: warning: format '%d' expects argument of type 'int', but argument 2 has type 'ssize_t' {aka 'long int'} [-Wformat=] 2020-05-18T21:41:42.720 INFO:tasks.workunit.client.0.smithi196.stderr: 180 | printf("Wrote %d instead of %d bytes.\n" 2020-05-18T21:41:42.720 INFO:tasks.workunit.client.0.smithi196.stderr: | ~^ 2020-05-18T21:41:42.720 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:42.720 INFO:tasks.workunit.client.0.smithi196.stderr: | int 2020-05-18T21:41:42.721 INFO:tasks.workunit.client.0.smithi196.stderr: | %ld 2020-05-18T21:41:42.721 INFO:tasks.workunit.client.0.smithi196.stderr: 181 | "Probably out of disk space\n", realsize, size); 2020-05-18T21:41:42.721 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~~~~~~ 2020-05-18T21:41:42.721 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:42.721 INFO:tasks.workunit.client.0.smithi196.stderr: | ssize_t {aka long int} 2020-05-18T21:41:42.722 INFO:tasks.workunit.client.0.smithi196.stderr:fh.c: In function 'fhseek': 2020-05-18T21:41:42.722 INFO:tasks.workunit.client.0.smithi196.stderr:fh.c:212:41: warning: format '%lld' expects argument of type 'long long int', but argument 3 has type 'uint64_t' {aka 'long unsigned int'} [-Wformat=] 2020-05-18T21:41:42.722 INFO:tasks.workunit.client.0.smithi196.stderr: 212 | fprintf(stderr, "tried to seek to %lld\n", offset); 2020-05-18T21:41:42.722 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~^ ~~~~~~ 2020-05-18T21:41:42.722 INFO:tasks.workunit.client.0.smithi196.stderr: | | | 2020-05-18T21:41:42.723 INFO:tasks.workunit.client.0.smithi196.stderr: | | uint64_t {aka long unsigned int} 2020-05-18T21:41:42.723 INFO:tasks.workunit.client.0.smithi196.stderr: | long long int 2020-05-18T21:41:42.723 INFO:tasks.workunit.client.0.smithi196.stderr: | %ld 2020-05-18T21:41:42.723 INFO:tasks.workunit.client.0.smithi196.stderr:fh.c:214:20: warning: format '%lld' expects argument of type 'long long int', but argument 3 has type 'uint64_t' {aka 'long unsigned int'} [-Wformat=] 2020-05-18T21:41:42.723 INFO:tasks.workunit.client.0.smithi196.stderr: 214 | fprintf(stderr, "tried to seek from current " 2020-05-18T21:41:42.723 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:42.724 INFO:tasks.workunit.client.0.smithi196.stderr: 215 | "position to %lld\n", offset); 2020-05-18T21:41:42.724 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~~~~ 2020-05-18T21:41:42.724 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:42.724 INFO:tasks.workunit.client.0.smithi196.stderr: | uint64_t {aka long unsigned int} 2020-05-18T21:41:42.724 INFO:tasks.workunit.client.0.smithi196.stderr:fh.c:215:21: note: format string is defined here 2020-05-18T21:41:42.725 INFO:tasks.workunit.client.0.smithi196.stderr: 215 | "position to %lld\n", offset); 2020-05-18T21:41:42.725 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~^ 2020-05-18T21:41:42.725 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:42.725 INFO:tasks.workunit.client.0.smithi196.stderr: | long long int 2020-05-18T21:41:42.725 INFO:tasks.workunit.client.0.smithi196.stderr: | %ld 2020-05-18T21:41:42.838 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2020-05-18T21:41:42.842 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2020-05-18T21:41:43.095 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2020-05-18T21:41:43.107 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2020-05-18T21:41:43.268 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2020-05-18T21:41:43.283 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2020-05-18T21:41:43.384 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2020-05-18T21:41:43.401 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2020-05-18T21:41:43.533 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2020-05-18T21:41:43.551 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2020-05-18T21:41:43.809 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2020-05-18T21:41:43.824 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2020-05-18T21:41:43.917 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_tg.c: In function 'tg_print_config_helper': 2020-05-18T21:41:43.917 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_tg.c:274:35: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t' {aka 'long unsigned int'} [-Wformat=] 2020-05-18T21:41:43.918 INFO:tasks.workunit.client.0.smithi196.stderr: 274 | printf("\t read_size = %llu\t(%s)\n", tg->read_size, 2020-05-18T21:41:43.918 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~^ ~~~~~~~~~~~~~ 2020-05-18T21:41:43.918 INFO:tasks.workunit.client.0.smithi196.stderr: | | | 2020-05-18T21:41:43.918 INFO:tasks.workunit.client.0.smithi196.stderr: | | uint64_t {aka long unsigned int} 2020-05-18T21:41:43.919 INFO:tasks.workunit.client.0.smithi196.stderr: | long long unsigned int 2020-05-18T21:41:43.919 INFO:tasks.workunit.client.0.smithi196.stderr: | %lu 2020-05-18T21:41:43.919 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_tg.c:283:35: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t' {aka 'long unsigned int'} [-Wformat=] 2020-05-18T21:41:43.919 INFO:tasks.workunit.client.0.smithi196.stderr: 283 | printf("\t write_size = %llu\t(%s)\n", tg->write_size, 2020-05-18T21:41:43.919 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~^ ~~~~~~~~~~~~~~ 2020-05-18T21:41:43.920 INFO:tasks.workunit.client.0.smithi196.stderr: | | | 2020-05-18T21:41:43.920 INFO:tasks.workunit.client.0.smithi196.stderr: | | uint64_t {aka long unsigned int} 2020-05-18T21:41:43.920 INFO:tasks.workunit.client.0.smithi196.stderr: | long long unsigned int 2020-05-18T21:41:43.920 INFO:tasks.workunit.client.0.smithi196.stderr: | %lu 2020-05-18T21:41:44.049 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2020-05-18T21:41:44.062 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2020-05-18T21:41:44.171 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_fs.c: In function 'verify_file': 2020-05-18T21:41:44.172 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_fs.c:217:19: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t' {aka 'long unsigned int'} [-Wformat=] 2020-05-18T21:41:44.172 INFO:tasks.workunit.client.0.smithi196.stderr: 217 | printf("size %llu bytes for file %s is invalid\n", 2020-05-18T21:41:44.172 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~^ 2020-05-18T21:41:44.172 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:44.172 INFO:tasks.workunit.client.0.smithi196.stderr: | long long unsigned int 2020-05-18T21:41:44.173 INFO:tasks.workunit.client.0.smithi196.stderr: | %lu 2020-05-18T21:41:44.173 INFO:tasks.workunit.client.0.smithi196.stderr: 218 | filesize, fname); 2020-05-18T21:41:44.173 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~~~~~~ 2020-05-18T21:41:44.173 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:44.173 INFO:tasks.workunit.client.0.smithi196.stderr: | uint64_t {aka long unsigned int} 2020-05-18T21:41:44.174 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_fs.c: In function 'fs_print_config': 2020-05-18T21:41:44.175 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_fs.c:595:22: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t' {aka 'long unsigned int'} [-Wformat=] 2020-05-18T21:41:44.175 INFO:tasks.workunit.client.0.smithi196.stderr: 595 | printf("\t\t %12llu (%6s) -> %u (%.2f\%)\n", 2020-05-18T21:41:44.175 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~~~^ 2020-05-18T21:41:44.175 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:44.176 INFO:tasks.workunit.client.0.smithi196.stderr: | long long unsigned int 2020-05-18T21:41:44.176 INFO:tasks.workunit.client.0.smithi196.stderr: | %12lu 2020-05-18T21:41:44.176 INFO:tasks.workunit.client.0.smithi196.stderr: 596 | fs->size_weights[i].size, 2020-05-18T21:41:44.176 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:44.176 INFO:tasks.workunit.client.0.smithi196.stderr: | | 2020-05-18T21:41:44.176 INFO:tasks.workunit.client.0.smithi196.stderr: | uint64_t {aka long unsigned int} 2020-05-18T21:41:44.177 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_fs.c:595:43: warning: unknown conversion type character ')' in format [-Wformat=] 2020-05-18T21:41:44.177 INFO:tasks.workunit.client.0.smithi196.stderr: 595 | printf("\t\t %12llu (%6s) -> %u (%.2f\%)\n", 2020-05-18T21:41:44.177 INFO:tasks.workunit.client.0.smithi196.stderr: | ^ 2020-05-18T21:41:44.177 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_fs.c:603:36: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t' {aka 'long unsigned int'} [-Wformat=] 2020-05-18T21:41:44.177 INFO:tasks.workunit.client.0.smithi196.stderr: 603 | printf("\t min file size = %llu\t(%s)\n", fs->minfilesize, 2020-05-18T21:41:44.178 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~^ ~~~~~~~~~~~~~~~ 2020-05-18T21:41:44.178 INFO:tasks.workunit.client.0.smithi196.stderr: | | | 2020-05-18T21:41:44.178 INFO:tasks.workunit.client.0.smithi196.stderr: | | uint64_t {aka long unsigned int} 2020-05-18T21:41:44.178 INFO:tasks.workunit.client.0.smithi196.stderr: | long long unsigned int 2020-05-18T21:41:44.178 INFO:tasks.workunit.client.0.smithi196.stderr: | %lu 2020-05-18T21:41:44.179 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_fs.c:605:36: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t' {aka 'long unsigned int'} [-Wformat=] 2020-05-18T21:41:44.179 INFO:tasks.workunit.client.0.smithi196.stderr: 605 | printf("\t max file size = %llu\t(%s)\n", fs->maxfilesize, 2020-05-18T21:41:44.179 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~^ ~~~~~~~~~~~~~~~ 2020-05-18T21:41:44.179 INFO:tasks.workunit.client.0.smithi196.stderr: | | | 2020-05-18T21:41:44.179 INFO:tasks.workunit.client.0.smithi196.stderr: | | uint64_t {aka long unsigned int} 2020-05-18T21:41:44.179 INFO:tasks.workunit.client.0.smithi196.stderr: | long long unsigned int 2020-05-18T21:41:44.180 INFO:tasks.workunit.client.0.smithi196.stderr: | %lu 2020-05-18T21:41:44.180 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_fs.c:616:41: warning: unknown conversion type character '\x0a' in format [-Wformat=] 2020-05-18T21:41:44.180 INFO:tasks.workunit.client.0.smithi196.stderr: 616 | printf("\t current utilization = %.2f\%\n", getfsutil(fs->basedir)*100); 2020-05-18T21:41:44.180 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~ 2020-05-18T21:41:44.181 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_fs.c:618:42: warning: unknown conversion type character '\x0a' in format [-Wformat=] 2020-05-18T21:41:44.181 INFO:tasks.workunit.client.0.smithi196.stderr: 618 | printf("\t desired utilization = %.2lf%\n", fs->desired_fsutil * 100); 2020-05-18T21:41:44.181 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~ 2020-05-18T21:41:44.181 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_fs.c: In function 'fs_needs_stats': 2020-05-18T21:41:44.181 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_fs.c:627:24: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2020-05-18T21:41:44.181 INFO:tasks.workunit.client.0.smithi196.stderr: 627 | return (fs != NULL) ? (int)fs->fsd.config : 0; 2020-05-18T21:41:44.182 INFO:tasks.workunit.client.0.smithi196.stderr: | ^ 2020-05-18T21:41:44.339 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2020-05-18T21:41:44.355 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2020-05-18T21:41:44.513 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2020-05-18T21:41:44.531 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2020-05-18T21:41:44.639 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_op.c: In function 'print_op_results': 2020-05-18T21:41:44.640 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_op.c:83:23: warning: embedded '\0' in format [-Wformat-contains-nul] 2020-05-18T21:41:44.640 INFO:tasks.workunit.client.0.smithi196.stderr: 83 | sprintf(buf, "%s/sec\0", buf); 2020-05-18T21:41:44.640 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~ 2020-05-18T21:41:44.640 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_op.c:86:19: warning: embedded '\0' in format [-Wformat-contains-nul] 2020-05-18T21:41:44.640 INFO:tasks.workunit.client.0.smithi196.stderr: 86 | sprintf(buf, "NA\0"); 2020-05-18T21:41:44.641 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~ 2020-05-18T21:41:44.641 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_op.c: In function 'print_results': 2020-05-18T21:41:44.641 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_op.c:112:61: warning: unknown conversion type character 'T' in format [-Wformat=] 2020-05-18T21:41:44.641 INFO:tasks.workunit.client.0.smithi196.stderr: 112 | printf(" Op Name Transactions\t Trans/sec\t% Trans\t % Op Weight\t Throughput\n"); 2020-05-18T21:41:44.641 INFO:tasks.workunit.client.0.smithi196.stderr: | ^ 2020-05-18T21:41:44.642 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_op.c:112:74: warning: unknown conversion type character 'O' in format [-Wformat=] 2020-05-18T21:41:44.642 INFO:tasks.workunit.client.0.smithi196.stderr: 112 | printf(" Op Name Transactions\t Trans/sec\t% Trans\t % Op Weight\t Throughput\n"); 2020-05-18T21:41:44.642 INFO:tasks.workunit.client.0.smithi196.stderr: | ^ 2020-05-18T21:41:44.658 INFO:tasks.workunit.client.0.smithi196.stderr:ffsb_op.c:83:19: warning: '/sec' directive writing 4 bytes into a region of size between 1 and 256 [-Wformat-overflow=] 2020-05-18T21:41:44.658 INFO:tasks.workunit.client.0.smithi196.stderr: 83 | sprintf(buf, "%s/sec\0", buf); 2020-05-18T21:41:44.659 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~ 2020-05-18T21:41:44.659 INFO:tasks.workunit.client.0.smithi196.stderr:In file included from /usr/include/stdio.h:867, 2020-05-18T21:41:44.659 INFO:tasks.workunit.client.0.smithi196.stderr: from ffsb_op.c:20: 2020-05-18T21:41:44.659 INFO:tasks.workunit.client.0.smithi196.stderr:/usr/include/x86_64-linux-gnu/bits/stdio2.h:36:10: note: '__builtin___sprintf_chk' output between 5 and 260 bytes into a destination of size 256 2020-05-18T21:41:44.659 INFO:tasks.workunit.client.0.smithi196.stderr: 36 | return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, 2020-05-18T21:41:44.660 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:44.660 INFO:tasks.workunit.client.0.smithi196.stderr: 37 | __bos (__s), __fmt, __va_arg_pack ()); 2020-05-18T21:41:44.660 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:44.718 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2020-05-18T21:41:44.732 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2020-05-18T21:41:44.833 INFO:tasks.workunit.client.0.smithi196.stderr:util.c: In function 'ffsb_bench_gettimeofday': 2020-05-18T21:41:44.833 INFO:tasks.workunit.client.0.smithi196.stderr:util.c:333:46: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t' {aka 'long unsigned int'} [-Wformat=] 2020-05-18T21:41:44.833 INFO:tasks.workunit.client.0.smithi196.stderr: 333 | printf("average time for gettimeofday(): %llu nsec\n", average); 2020-05-18T21:41:44.834 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~^ ~~~~~~~ 2020-05-18T21:41:44.834 INFO:tasks.workunit.client.0.smithi196.stderr: | | | 2020-05-18T21:41:44.834 INFO:tasks.workunit.client.0.smithi196.stderr: | | uint64_t {aka long unsigned int} 2020-05-18T21:41:44.834 INFO:tasks.workunit.client.0.smithi196.stderr: | long long unsigned int 2020-05-18T21:41:44.834 INFO:tasks.workunit.client.0.smithi196.stderr: | %lu 2020-05-18T21:41:44.835 INFO:tasks.workunit.client.0.smithi196.stderr:util.c: In function 'ffsb_bench_getpid': 2020-05-18T21:41:44.835 INFO:tasks.workunit.client.0.smithi196.stderr:util.c:350:40: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type 'uint64_t' {aka 'long unsigned int'} [-Wformat=] 2020-05-18T21:41:44.835 INFO:tasks.workunit.client.0.smithi196.stderr: 350 | printf("average time for getpid(): %llu nsec\n", average); 2020-05-18T21:41:44.835 INFO:tasks.workunit.client.0.smithi196.stderr: | ~~~^ ~~~~~~~ 2020-05-18T21:41:44.835 INFO:tasks.workunit.client.0.smithi196.stderr: | | | 2020-05-18T21:41:44.836 INFO:tasks.workunit.client.0.smithi196.stderr: | | uint64_t {aka long unsigned int} 2020-05-18T21:41:44.836 INFO:tasks.workunit.client.0.smithi196.stderr: | long long unsigned int 2020-05-18T21:41:44.836 INFO:tasks.workunit.client.0.smithi196.stderr: | %lu 2020-05-18T21:41:44.947 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/util.Tpo .deps/util.Po 2020-05-18T21:41:44.961 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2020-05-18T21:41:45.064 INFO:tasks.workunit.client.0.smithi196.stderr:In file included from parser.h:22, 2020-05-18T21:41:45.065 INFO:tasks.workunit.client.0.smithi196.stderr: from parser.c:25: 2020-05-18T21:41:45.065 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c: In function 'init_filesys': 2020-05-18T21:41:45.067 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2020-05-18T21:41:45.068 INFO:tasks.workunit.client.0.smithi196.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2020-05-18T21:41:45.068 INFO:tasks.workunit.client.0.smithi196.stderr: | ^ 2020-05-18T21:41:45.068 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2020-05-18T21:41:45.068 INFO:tasks.workunit.client.0.smithi196.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2020-05-18T21:41:45.068 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~ 2020-05-18T21:41:45.069 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2020-05-18T21:41:45.069 INFO:tasks.workunit.client.0.smithi196.stderr: 36 | container_of(ptr, type, member) 2020-05-18T21:41:45.069 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~ 2020-05-18T21:41:45.069 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2020-05-18T21:41:45.069 INFO:tasks.workunit.client.0.smithi196.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2020-05-18T21:41:45.070 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~ 2020-05-18T21:41:45.070 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c:934:3: note: in expansion of macro 'list_for_each_entry' 2020-05-18T21:41:45.070 INFO:tasks.workunit.client.0.smithi196.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2020-05-18T21:41:45.070 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:45.070 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2020-05-18T21:41:45.071 INFO:tasks.workunit.client.0.smithi196.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2020-05-18T21:41:45.071 INFO:tasks.workunit.client.0.smithi196.stderr: | ^ 2020-05-18T21:41:45.071 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2020-05-18T21:41:45.071 INFO:tasks.workunit.client.0.smithi196.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2020-05-18T21:41:45.071 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~ 2020-05-18T21:41:45.072 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2020-05-18T21:41:45.072 INFO:tasks.workunit.client.0.smithi196.stderr: 36 | container_of(ptr, type, member) 2020-05-18T21:41:45.072 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~ 2020-05-18T21:41:45.072 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2020-05-18T21:41:45.072 INFO:tasks.workunit.client.0.smithi196.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2020-05-18T21:41:45.073 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~ 2020-05-18T21:41:45.073 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c:934:3: note: in expansion of macro 'list_for_each_entry' 2020-05-18T21:41:45.073 INFO:tasks.workunit.client.0.smithi196.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2020-05-18T21:41:45.073 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:45.074 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2020-05-18T21:41:45.074 INFO:tasks.workunit.client.0.smithi196.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2020-05-18T21:41:45.074 INFO:tasks.workunit.client.0.smithi196.stderr: | ^ 2020-05-18T21:41:45.074 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2020-05-18T21:41:45.075 INFO:tasks.workunit.client.0.smithi196.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2020-05-18T21:41:45.075 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~ 2020-05-18T21:41:45.075 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2020-05-18T21:41:45.075 INFO:tasks.workunit.client.0.smithi196.stderr: 36 | container_of(ptr, type, member) 2020-05-18T21:41:45.075 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~ 2020-05-18T21:41:45.076 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2020-05-18T21:41:45.076 INFO:tasks.workunit.client.0.smithi196.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2020-05-18T21:41:45.076 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~ 2020-05-18T21:41:45.076 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c:941:3: note: in expansion of macro 'list_for_each_entry' 2020-05-18T21:41:45.076 INFO:tasks.workunit.client.0.smithi196.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2020-05-18T21:41:45.077 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:45.077 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2020-05-18T21:41:45.077 INFO:tasks.workunit.client.0.smithi196.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2020-05-18T21:41:45.077 INFO:tasks.workunit.client.0.smithi196.stderr: | ^ 2020-05-18T21:41:45.077 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2020-05-18T21:41:45.078 INFO:tasks.workunit.client.0.smithi196.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2020-05-18T21:41:45.078 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~ 2020-05-18T21:41:45.078 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2020-05-18T21:41:45.078 INFO:tasks.workunit.client.0.smithi196.stderr: 36 | container_of(ptr, type, member) 2020-05-18T21:41:45.078 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~ 2020-05-18T21:41:45.079 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2020-05-18T21:41:45.080 INFO:tasks.workunit.client.0.smithi196.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2020-05-18T21:41:45.080 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~ 2020-05-18T21:41:45.080 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c:941:3: note: in expansion of macro 'list_for_each_entry' 2020-05-18T21:41:45.080 INFO:tasks.workunit.client.0.smithi196.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2020-05-18T21:41:45.081 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:45.081 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c: In function 'init_tg_stats': 2020-05-18T21:41:45.081 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2020-05-18T21:41:45.081 INFO:tasks.workunit.client.0.smithi196.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2020-05-18T21:41:45.081 INFO:tasks.workunit.client.0.smithi196.stderr: | ^ 2020-05-18T21:41:45.082 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2020-05-18T21:41:45.082 INFO:tasks.workunit.client.0.smithi196.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2020-05-18T21:41:45.082 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~ 2020-05-18T21:41:45.082 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2020-05-18T21:41:45.083 INFO:tasks.workunit.client.0.smithi196.stderr: 36 | container_of(ptr, type, member) 2020-05-18T21:41:45.083 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~ 2020-05-18T21:41:45.083 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2020-05-18T21:41:45.083 INFO:tasks.workunit.client.0.smithi196.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2020-05-18T21:41:45.083 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~ 2020-05-18T21:41:45.084 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c:971:6: note: in expansion of macro 'list_for_each_entry' 2020-05-18T21:41:45.084 INFO:tasks.workunit.client.0.smithi196.stderr: 971 | list_for_each_entry(tmp_list, 2020-05-18T21:41:45.084 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:45.084 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2020-05-18T21:41:45.084 INFO:tasks.workunit.client.0.smithi196.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2020-05-18T21:41:45.085 INFO:tasks.workunit.client.0.smithi196.stderr: | ^ 2020-05-18T21:41:45.085 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2020-05-18T21:41:45.085 INFO:tasks.workunit.client.0.smithi196.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2020-05-18T21:41:45.085 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~ 2020-05-18T21:41:45.085 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2020-05-18T21:41:45.086 INFO:tasks.workunit.client.0.smithi196.stderr: 36 | container_of(ptr, type, member) 2020-05-18T21:41:45.086 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~ 2020-05-18T21:41:45.086 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2020-05-18T21:41:45.086 INFO:tasks.workunit.client.0.smithi196.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2020-05-18T21:41:45.086 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~ 2020-05-18T21:41:45.086 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c:971:6: note: in expansion of macro 'list_for_each_entry' 2020-05-18T21:41:45.087 INFO:tasks.workunit.client.0.smithi196.stderr: 971 | list_for_each_entry(tmp_list, 2020-05-18T21:41:45.087 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:45.087 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2020-05-18T21:41:45.087 INFO:tasks.workunit.client.0.smithi196.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2020-05-18T21:41:45.087 INFO:tasks.workunit.client.0.smithi196.stderr: | ^ 2020-05-18T21:41:45.088 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2020-05-18T21:41:45.088 INFO:tasks.workunit.client.0.smithi196.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2020-05-18T21:41:45.088 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~ 2020-05-18T21:41:45.088 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2020-05-18T21:41:45.088 INFO:tasks.workunit.client.0.smithi196.stderr: 36 | container_of(ptr, type, member) 2020-05-18T21:41:45.089 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~ 2020-05-18T21:41:45.089 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2020-05-18T21:41:45.089 INFO:tasks.workunit.client.0.smithi196.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2020-05-18T21:41:45.089 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~ 2020-05-18T21:41:45.089 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c:980:6: note: in expansion of macro 'list_for_each_entry' 2020-05-18T21:41:45.090 INFO:tasks.workunit.client.0.smithi196.stderr: 980 | list_for_each_entry(tmp_list, 2020-05-18T21:41:45.090 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:45.091 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2020-05-18T21:41:45.091 INFO:tasks.workunit.client.0.smithi196.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2020-05-18T21:41:45.091 INFO:tasks.workunit.client.0.smithi196.stderr: | ^ 2020-05-18T21:41:45.091 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2020-05-18T21:41:45.091 INFO:tasks.workunit.client.0.smithi196.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2020-05-18T21:41:45.092 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~ 2020-05-18T21:41:45.092 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2020-05-18T21:41:45.092 INFO:tasks.workunit.client.0.smithi196.stderr: 36 | container_of(ptr, type, member) 2020-05-18T21:41:45.092 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~ 2020-05-18T21:41:45.092 INFO:tasks.workunit.client.0.smithi196.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2020-05-18T21:41:45.093 INFO:tasks.workunit.client.0.smithi196.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2020-05-18T21:41:45.093 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~ 2020-05-18T21:41:45.093 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c:980:6: note: in expansion of macro 'list_for_each_entry' 2020-05-18T21:41:45.093 INFO:tasks.workunit.client.0.smithi196.stderr: 980 | list_for_each_entry(tmp_list, 2020-05-18T21:41:45.093 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:45.142 INFO:tasks.workunit.client.0.smithi196.stderr:In file included from /usr/include/string.h:495, 2020-05-18T21:41:45.143 INFO:tasks.workunit.client.0.smithi196.stderr: from parser.c:19: 2020-05-18T21:41:45.143 INFO:tasks.workunit.client.0.smithi196.stderr:In function 'memcpy', 2020-05-18T21:41:45.143 INFO:tasks.workunit.client.0.smithi196.stderr: inlined from 'size64_convert' at parser.c:119:2: 2020-05-18T21:41:45.144 INFO:tasks.workunit.client.0.smithi196.stderr:/usr/include/x86_64-linux-gnu/bits/string_fortified.h:34:10: warning: '__builtin_memcpy' reading 3 bytes from a region of size 2 [-Wstringop-overflow=] 2020-05-18T21:41:45.144 INFO:tasks.workunit.client.0.smithi196.stderr: 34 | return __builtin___memcpy_chk (__dest, __src, __len, __bos0 (__dest)); 2020-05-18T21:41:45.144 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:45.210 INFO:tasks.workunit.client.0.smithi196.stderr:In function 'strncpy', 2020-05-18T21:41:45.211 INFO:tasks.workunit.client.0.smithi196.stderr: inlined from 'set_weight' at parser.c:623:4: 2020-05-18T21:41:45.211 INFO:tasks.workunit.client.0.smithi196.stderr:/usr/include/x86_64-linux-gnu/bits/string_fortified.h:106:10: warning: '__builtin_strncpy' specified bound depends on the length of the source argument [-Wstringop-overflow=] 2020-05-18T21:41:45.211 INFO:tasks.workunit.client.0.smithi196.stderr: 106 | return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest)); 2020-05-18T21:41:45.212 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:45.212 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c: In function 'set_weight': 2020-05-18T21:41:45.212 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c:620:10: note: length computed here 2020-05-18T21:41:45.212 INFO:tasks.workunit.client.0.smithi196.stderr: 620 | len = strlen(tmp_config->name); 2020-05-18T21:41:45.213 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:45.216 INFO:tasks.workunit.client.0.smithi196.stderr:In file included from /usr/include/string.h:495, 2020-05-18T21:41:45.216 INFO:tasks.workunit.client.0.smithi196.stderr: from parser.c:19: 2020-05-18T21:41:45.216 INFO:tasks.workunit.client.0.smithi196.stderr:In function 'strncpy', 2020-05-18T21:41:45.217 INFO:tasks.workunit.client.0.smithi196.stderr: inlined from 'get_weight_total' at parser.c:645:4: 2020-05-18T21:41:45.217 INFO:tasks.workunit.client.0.smithi196.stderr:/usr/include/x86_64-linux-gnu/bits/string_fortified.h:106:10: warning: '__builtin_strncpy' specified bound depends on the length of the source argument [-Wstringop-overflow=] 2020-05-18T21:41:45.217 INFO:tasks.workunit.client.0.smithi196.stderr: 106 | return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest)); 2020-05-18T21:41:45.218 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:45.218 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c: In function 'get_weight_total': 2020-05-18T21:41:45.219 INFO:tasks.workunit.client.0.smithi196.stderr:parser.c:642:10: note: length computed here 2020-05-18T21:41:45.219 INFO:tasks.workunit.client.0.smithi196.stderr: 642 | len = strlen(tmp_config->name); 2020-05-18T21:41:45.219 INFO:tasks.workunit.client.0.smithi196.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2020-05-18T21:41:45.389 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2020-05-18T21:41:45.405 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2020-05-18T21:41:45.576 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2020-05-18T21:41:45.583 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2020-05-18T21:41:45.757 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2020-05-18T21:41:45.772 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2020-05-18T21:41:45.850 INFO:tasks.workunit.client.0.smithi196.stdout:mv -f .deps/list.Tpo .deps/list.Po 2020-05-18T21:41:45.872 INFO:tasks.workunit.client.0.smithi196.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2020-05-18T21:41:46.180 INFO:tasks.workunit.client.0.smithi196.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb-6.0-rc2' 2020-05-18T21:41:46.227 INFO:tasks.workunit.client.0.smithi196.stdout:FFSB version 6.0-RC2 started 2020-05-18T21:41:46.228 INFO:tasks.workunit.client.0.smithi196.stdout: 2020-05-18T21:41:46.236 INFO:tasks.workunit.client.0.smithi196.stdout:benchmark time = 300 2020-05-18T21:41:46.236 INFO:tasks.workunit.client.0.smithi196.stdout:ThreadGroup 0 2020-05-18T21:41:46.237 INFO:tasks.workunit.client.0.smithi196.stdout:================ 2020-05-18T21:41:46.237 INFO:tasks.workunit.client.0.smithi196.stdout: num_threads = 32 2020-05-18T21:41:46.237 INFO:tasks.workunit.client.0.smithi196.stdout: 2020-05-18T21:41:46.237 INFO:tasks.workunit.client.0.smithi196.stdout: read_random = off 2020-05-18T21:41:46.238 INFO:tasks.workunit.client.0.smithi196.stdout: read_size = 0 (0B) 2020-05-18T21:41:46.238 INFO:tasks.workunit.client.0.smithi196.stdout: read_blocksize = 0 (0B) 2020-05-18T21:41:46.238 INFO:tasks.workunit.client.0.smithi196.stdout: read_skip = off 2020-05-18T21:41:46.238 INFO:tasks.workunit.client.0.smithi196.stdout: read_skipsize = 0 (0B) 2020-05-18T21:41:46.238 INFO:tasks.workunit.client.0.smithi196.stdout: 2020-05-18T21:41:46.239 INFO:tasks.workunit.client.0.smithi196.stdout: write_random = on 2020-05-18T21:41:46.239 INFO:tasks.workunit.client.0.smithi196.stdout: write_size = 5242880 (5MB) 2020-05-18T21:41:46.239 INFO:tasks.workunit.client.0.smithi196.stdout: fsync_file = 0 2020-05-18T21:41:46.239 INFO:tasks.workunit.client.0.smithi196.stdout: write_blocksize = 4096 (4KB) 2020-05-18T21:41:46.239 INFO:tasks.workunit.client.0.smithi196.stdout: wait time = 0 2020-05-18T21:41:46.240 INFO:tasks.workunit.client.0.smithi196.stdout: 2020-05-18T21:41:46.240 INFO:tasks.workunit.client.0.smithi196.stdout: op weights 2020-05-18T21:41:46.240 INFO:tasks.workunit.client.0.smithi196.stdout: read = 0 (0.00%) 2020-05-18T21:41:46.240 INFO:tasks.workunit.client.0.smithi196.stdout: readall = 0 (0.00%) 2020-05-18T21:41:46.241 INFO:tasks.workunit.client.0.smithi196.stdout: write = 1 (100.00%) 2020-05-18T21:41:46.241 INFO:tasks.workunit.client.0.smithi196.stdout: create = 0 (0.00%) 2020-05-18T21:41:46.241 INFO:tasks.workunit.client.0.smithi196.stdout: append = 0 (0.00%) 2020-05-18T21:41:46.241 INFO:tasks.workunit.client.0.smithi196.stdout: delete = 0 (0.00%) 2020-05-18T21:41:46.242 INFO:tasks.workunit.client.0.smithi196.stdout: metaop = 0 (0.00%) 2020-05-18T21:41:46.242 INFO:tasks.workunit.client.0.smithi196.stdout: createdir = 0 (0.00%) 2020-05-18T21:41:46.242 INFO:tasks.workunit.client.0.smithi196.stdout: stat = 0 (0.00%) 2020-05-18T21:41:46.242 INFO:tasks.workunit.client.0.smithi196.stdout: writeall = 0 (0.00%) 2020-05-18T21:41:46.243 INFO:tasks.workunit.client.0.smithi196.stdout: writeall_fsync = 0 (0.00%) 2020-05-18T21:41:46.243 INFO:tasks.workunit.client.0.smithi196.stdout: open_close = 0 (0.00%) 2020-05-18T21:41:46.243 INFO:tasks.workunit.client.0.smithi196.stdout: write_fsync = 0 (0.00%) 2020-05-18T21:41:46.243 INFO:tasks.workunit.client.0.smithi196.stdout: create_fsync = 0 (0.00%) 2020-05-18T21:41:46.243 INFO:tasks.workunit.client.0.smithi196.stdout: append_fsync = 0 (0.00%) 2020-05-18T21:41:46.244 INFO:tasks.workunit.client.0.smithi196.stdout: 2020-05-18T21:41:46.244 INFO:tasks.workunit.client.0.smithi196.stdout:FileSystem . 2020-05-18T21:41:46.244 INFO:tasks.workunit.client.0.smithi196.stdout:========== 2020-05-18T21:41:46.244 INFO:tasks.workunit.client.0.smithi196.stdout: num_dirs = 0 2020-05-18T21:41:46.244 INFO:tasks.workunit.client.0.smithi196.stdout: starting files = 128 2020-05-18T21:41:46.245 INFO:tasks.workunit.client.0.smithi196.stdout: 2020-05-18T21:41:46.245 INFO:tasks.workunit.client.0.smithi196.stdout: min file size = 104857600 (100MB) 2020-05-18T21:41:46.245 INFO:tasks.workunit.client.0.smithi196.stdout: max file size = 104857600 (100MB) 2020-05-18T21:41:46.245 INFO:tasks.workunit.client.0.smithi196.stdout: directio = off 2020-05-18T21:41:46.246 INFO:tasks.workunit.client.0.smithi196.stdout: alignedio = on 2020-05-18T21:41:46.246 INFO:tasks.workunit.client.0.smithi196.stdout: bufferedio = off 2020-05-18T21:41:46.246 INFO:tasks.workunit.client.0.smithi196.stdout: 2020-05-18T21:41:46.246 INFO:tasks.workunit.client.0.smithi196.stdout: aging is off 2020-05-18T21:41:46.247 INFO:tasks.workunit.client.0.smithi196.stdout: current utilization = 0.00% 2020-05-18T21:41:46.247 INFO:tasks.workunit.client.0.smithi196.stdout: 2020-05-18T21:41:46.248 INFO:tasks.workunit.client.0.smithi196.stdout:checking existing fs: . 2020-05-18T21:41:46.248 INFO:tasks.workunit.client.0.smithi196.stdout:recreating new fileset 2020-05-18T21:41:46.248 INFO:tasks.workunit.client.0.smithi196.stderr:opendir: No such file or directory 2020-05-18T21:42:11.810 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:42:11.820 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:42:42.159 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:42:42.163 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:43:12.212 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:43:12.216 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:43:42.265 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:43:42.269 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:44:12.319 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:44:12.322 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:44:42.656 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:44:42.659 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:45:12.768 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:45:12.772 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:45:42.863 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:45:42.866 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:46:12.908 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:46:12.912 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:46:42.969 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:46:42.972 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:47:13.023 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:47:13.027 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:47:43.085 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:47:43.088 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:48:13.163 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:48:13.167 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:48:43.217 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:48:43.221 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:49:13.271 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:49:13.276 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:49:43.343 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:49:43.354 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:50:13.705 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:50:13.708 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:50:43.746 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:50:43.749 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:51:13.818 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:51:13.822 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:51:43.872 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:51:43.874 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:52:13.925 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:52:13.928 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:52:43.979 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:52:43.982 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:53:14.031 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:53:14.035 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:53:44.080 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:53:44.083 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:54:14.130 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:54:14.134 INFO:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:54:44.197 INFO:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-05-18T21:54:58.310 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:54:58.305+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:54:58.310 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:54:58.305+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:54:58.311 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:54:58.305+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:54:58.311 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:54:58.305+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:54:58.313 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:54:58.309+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:54:58.313 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:54:58.309+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:54:58.313 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:54:58.309+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:54:58.313 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:54:58.309+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:54:58.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:54:58.309+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:54:58.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:54:58.309+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:54:58.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:54:58.309+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:54:58.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:54:58.309+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:54:58.315 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:54:58.313+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:54:58.315 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:54:58.313+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:54:58.315 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:54:58.313+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:54:58.315 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:54:58.313+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:54:59.269 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:54:59.269+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:54:59.269 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:54:59.269+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:54:59.269 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:54:59.269+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:54:59.270 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:54:59.269+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:54:59.302 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:54:59.301+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:54:59.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:54:59.301+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:54:59.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:54:59.301+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:54:59.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:54:59.301+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:54:59.337 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:54:59.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:54:59.337 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:54:59.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:54:59.338 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:54:59.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:54:59.338 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:54:59.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:54:59.338 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:54:59.337+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:54:59.338 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:54:59.337+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:54:59.339 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:54:59.337+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:54:59.339 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:54:59.337+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:00.276 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:00.277+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:00.277 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:00.277+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:00.277 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:00.277+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:00.277 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:00.277+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:00.294 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:00.293+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:00.295 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:00.293+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:00.295 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:00.293+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:00.295 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:00.293+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:00.311 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:00.309+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:00.311 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:00.309+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:00.311 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:00.309+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:00.311 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:00.309+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:00.374 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:00.373+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:00.375 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:00.373+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:00.375 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:00.373+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:00.375 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:00.373+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:01.241 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:01.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:01.241 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:01.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:01.241 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:01.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:01.242 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:01.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:01.315 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:01.313+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:01.315 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:01.313+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:01.315 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:01.313+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:01.315 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:01.313+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:01.333 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:01.333+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:01.334 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:01.333+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:01.334 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:01.333+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:01.334 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:01.333+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:01.339 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:01.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:01.339 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:01.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:01.339 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:01.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:01.340 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:01.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:02.275 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:02.273+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:02.275 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:02.273+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:02.276 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:02.273+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:02.276 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:02.273+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:02.310 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:02.305+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:02.310 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:02.305+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:02.310 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:02.305+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:02.310 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:02.305+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:02.347 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:02.345+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:02.347 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:02.345+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:02.347 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:02.345+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:02.347 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:02.345+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:02.356 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:02.353+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:02.356 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:02.353+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:02.356 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:02.353+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:02.356 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:02.353+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:03.229 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:03.229+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:03.229 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:03.229+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:03.230 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:03.229+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:03.230 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:03.229+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:03.288 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:03.285+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:03.288 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:03.285+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:03.289 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:03.285+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:03.289 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:03.285+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:03.324 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:03.325+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:03.325 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:03.325+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:03.325 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:03.325+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:03.326 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:03.325+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:03.372 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:03.369+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:03.372 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:03.369+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:03.372 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:03.369+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:03.372 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:03.369+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:04.235 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:04.233+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:04.236 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:04.233+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:04.236 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:04.233+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:04.236 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:04.233+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:04.337 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:04.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:04.338 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:04.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:04.338 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:04.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:04.338 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:04.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:04.342 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:04.341+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:04.342 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:04.341+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:04.343 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:04.341+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:04.343 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:04.341+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:04.377 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:04.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:04.377 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:04.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:04.378 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:04.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:04.378 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:04.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:05.224 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:05.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:05.225 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:05.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:05.225 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:05.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:05.225 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:05.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:05.313 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:05.313+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:05.314 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:05.313+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:05.314 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:05.313+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:05.314 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:05.313+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:05.329 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:05.329+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:05.329 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:05.329+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:05.330 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:05.329+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:05.330 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:05.329+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:05.384 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:05.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:05.384 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:05.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:05.384 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:05.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:05.385 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:05.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:06.243 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:06.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:06.243 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:06.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:06.243 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:06.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:06.243 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:06.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:06.325 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:06.325+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:06.325 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:06.325+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:06.325 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:06.325+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:06.326 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:06.325+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:06.355 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:06.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:06.355 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:06.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:06.355 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:06.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:06.355 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:06.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:06.373 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:06.373+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:06.374 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:06.373+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:06.374 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:06.373+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:06.374 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:06.373+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:07.225 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:07.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:07.225 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:07.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:07.225 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:07.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:07.225 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:07.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:07.339 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:07.337+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:07.339 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:07.337+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:07.339 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:07.337+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:07.339 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:07.337+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:07.341 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:07.341+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:07.341 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:07.341+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:07.341 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:07.341+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:07.342 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:07.341+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:07.346 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:07.345+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:07.346 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:07.345+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:07.347 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:07.345+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:07.347 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:07.345+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:08.183 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:08.181+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:08.184 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:08.181+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:08.184 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:08.181+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:08.184 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:08.181+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:08.333 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:08.333+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:08.333 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:08.333+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:08.334 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:08.333+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:08.334 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:08.333+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:08.374 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:08.373+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:08.374 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:08.373+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:08.374 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:08.373+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:08.374 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:08.373+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:08.379 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:08.377+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:08.379 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:08.377+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:08.379 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:08.377+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:08.380 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:08.377+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:09.226 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:09.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:09.226 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:09.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:09.226 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:09.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:09.227 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:09.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:09.368 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:09.369+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:09.368 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:09.369+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:09.368 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:09.369+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:09.368 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:09.369+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:09.378 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:09.377+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:09.378 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:09.377+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:09.378 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:09.377+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:09.378 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:09.377+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:09.422 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:09.421+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:09.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:09.421+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:09.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:09.421+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:09.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:09.421+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:10.177 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:10.177+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:10.178 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:10.177+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:10.178 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:10.177+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:10.178 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:10.177+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:10.330 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:10.329+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:10.330 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:10.329+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:10.330 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:10.329+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:10.331 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:10.329+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:10.337 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:10.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:10.338 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:10.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:10.338 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:10.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:10.338 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:10.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:10.377 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:10.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:10.377 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:10.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:10.377 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:10.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:10.378 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:10.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:11.195 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:11.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:11.195 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:11.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:11.196 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:11.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:11.196 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:11.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:11.309 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:11.309+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:11.309 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:11.309+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:11.310 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:11.309+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:11.310 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:11.309+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:11.344 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:11.345+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:11.344 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:11.345+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:11.345 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:11.345+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:11.345 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:11.345+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:11.347 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:11.345+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:11.348 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:11.345+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:11.348 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:11.345+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:11.348 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:11.345+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:12.219 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:12.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:12.219 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:12.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:12.219 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:12.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:12.219 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:12.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:12.288 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:12.289+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:12.288 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:12.289+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:12.289 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:12.289+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:12.289 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:12.289+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:12.304 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:12.305+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:12.304 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:12.305+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:12.305 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:12.305+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:12.305 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:12.305+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:12.369 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:12.369+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:12.370 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:12.369+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:12.370 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:12.369+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:12.370 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:12.369+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:13.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:13.201+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:13.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:13.201+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:13.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:13.201+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:13.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:13.201+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:13.267 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:13.265+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:13.268 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:13.265+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:13.268 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:13.265+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:13.268 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:13.265+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:13.313 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:13.313+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:13.314 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:13.313+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:13.314 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:13.313+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:13.314 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:13.313+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:13.360 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:13.361+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:13.361 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:13.361+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:13.361 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:13.361+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:13.361 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:13.361+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:14.244 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:14.245+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:14.244 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:14.245+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:14.245 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:14.245+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:14.245 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:14.245+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:14.283 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:14.281+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:14.283 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:14.281+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:14.283 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:14.281+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:14.283 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:14.281+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:14.298 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:14.297+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:14.298 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:14.297+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:14.298 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:14.297+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:14.298 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:14.297+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:14.379 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:14.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:14.379 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:14.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:14.380 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:14.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:14.380 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:14.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:15.201 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:15.201+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:15.201 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:15.201+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:15.202 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:15.201+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:15.202 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:15.201+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:15.277 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:15.277+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:15.278 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:15.277+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:15.278 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:15.277+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:15.278 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:15.277+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:15.320 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:15.317+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:15.320 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:15.317+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:15.320 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:15.317+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:15.321 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:15.317+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:15.417 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:15.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:15.418 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:15.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:15.418 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:15.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:15.418 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:15.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:16.179 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:16.177+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:16.179 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:16.177+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:16.180 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:16.177+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:16.180 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:16.177+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:16.274 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:16.273+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:16.275 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:16.273+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:16.275 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:16.273+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:16.275 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:16.273+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:16.366 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:16.365+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:16.366 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:16.365+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:16.367 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:16.365+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:16.367 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:16.365+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:16.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:16.421+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:16.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:16.421+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:16.424 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:16.421+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:16.424 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:16.421+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:17.197 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:17.197+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:17.198 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:17.197+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:17.198 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:17.197+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:17.198 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:17.197+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:17.280 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:17.281+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:17.280 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:17.281+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:17.281 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:17.281+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:17.281 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:17.281+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:17.355 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:17.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:17.355 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:17.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:17.356 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:17.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:17.356 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:17.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:17.433 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:17.433+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:17.434 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:17.433+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:17.434 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:17.433+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:17.434 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:17.433+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:18.192 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:18.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:18.192 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:18.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:18.193 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:18.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:18.193 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:18.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:18.319 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:18.317+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:18.320 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:18.317+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:18.320 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:18.317+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:18.320 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:18.317+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:18.375 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:18.373+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:18.375 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:18.373+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:18.375 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:18.373+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:18.375 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:18.373+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:18.457 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:18.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:18.458 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:18.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:18.458 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:18.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:18.458 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:18.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:19.163 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:19.161+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:19.164 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:19.161+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:19.164 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:19.161+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:19.164 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:19.161+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:19.306 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:19.305+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:19.306 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:19.305+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:19.307 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:19.305+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:19.307 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:19.305+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:19.397 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:19.397+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:19.397 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:19.397+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:19.397 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:19.397+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:19.398 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:19.397+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:19.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:19.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:19.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:19.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:19.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:19.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:19.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:19.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:20.168 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:20.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:20.169 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:20.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:20.169 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:20.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:20.169 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:20.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:20.351 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:20.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:20.352 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:20.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:20.352 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:20.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:20.352 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:20.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:20.355 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:20.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:20.355 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:20.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:20.355 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:20.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:20.356 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:20.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:20.485 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:20.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:20.485 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:20.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:20.485 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:20.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:20.486 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:20.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:21.206 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:21.205+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:21.207 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:21.205+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:21.207 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:21.205+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:21.207 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:21.205+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:21.334 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:21.333+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:21.334 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:21.333+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:21.334 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:21.333+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:21.335 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:21.333+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:21.349 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:21.349+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:21.349 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:21.349+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:21.350 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:21.349+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:21.350 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:21.349+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:21.509 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:21.509+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:21.509 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:21.509+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:21.510 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:21.509+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:21.510 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:21.509+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:22.172 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:22.173+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:22.172 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:22.173+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:22.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:22.173+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:22.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:22.173+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:22.318 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:22.317+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:22.318 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:22.317+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:22.319 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:22.317+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:22.319 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:22.317+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:22.353 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:22.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:22.353 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:22.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:22.354 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:22.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:22.354 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:22.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:22.528 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:22.525+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:22.528 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:22.525+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:22.528 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:22.525+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:22.529 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:22.525+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:23.215 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:23.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:23.216 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:23.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:23.216 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:23.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:23.216 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:23.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:23.342 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:23.341+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:23.343 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:23.341+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:23.343 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:23.341+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:23.343 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:23.341+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:23.400 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:23.401+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:23.400 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:23.401+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:23.400 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:23.401+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:23.401 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:23.401+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:23.542 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:23.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:23.543 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:23.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:23.543 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:23.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:23.543 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:23.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:24.210 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:24.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:24.210 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:24.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:24.211 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:24.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:24.211 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:24.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:24.389 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:24.389+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:24.389 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:24.389+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:24.390 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:24.389+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:24.390 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:24.389+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:24.427 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:24.425+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:24.427 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:24.425+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:24.427 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:24.425+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:24.427 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:24.425+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:24.570 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:24.569+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:24.571 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:24.569+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:24.571 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:24.569+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:24.571 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:24.569+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:25.228 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:25.229+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:25.229 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:25.229+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:25.229 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:25.229+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:25.229 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:25.229+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:25.380 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:25.381+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:25.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:25.381+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:25.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:25.381+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:25.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:25.381+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:25.398 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:25.397+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:25.398 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:25.397+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:25.398 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:25.397+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:25.399 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:25.397+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:25.563 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:25.561+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:25.563 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:25.561+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:25.563 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:25.561+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:25.563 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:25.561+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:26.264 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:26.265+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:26.264 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:26.265+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:26.265 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:26.265+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:26.265 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:26.265+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:26.399 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:26.397+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:26.400 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:26.397+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:26.400 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:26.397+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:26.400 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:26.397+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:26.413 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:26.413+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:26.414 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:26.413+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:26.414 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:26.413+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:26.414 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:26.413+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:26.574 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:26.573+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:26.574 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:26.573+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:26.575 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:26.573+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:26.575 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:26.573+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:27.232 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:27.233+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:27.232 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:27.233+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:27.233 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:27.233+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:27.233 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:27.233+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:27.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:27.393+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:27.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:27.393+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:27.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:27.393+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:27.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:27.393+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:27.406 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:27.405+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:27.406 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:27.405+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:27.407 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:27.405+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:27.407 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:27.405+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:27.527 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:27.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:27.528 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:27.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:27.528 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:27.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:27.528 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:27.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:28.193 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:28.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:28.193 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:28.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:28.193 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:28.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:28.194 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:28.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:28.355 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:28.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:28.355 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:28.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:28.356 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:28.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:28.356 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:28.353+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:28.407 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:28.405+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:28.408 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:28.405+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:28.408 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:28.405+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:28.408 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:28.405+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:28.556 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:28.553+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:28.556 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:28.553+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:28.557 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:28.553+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:28.557 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:28.553+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:29.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:29.173+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:29.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:29.173+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:29.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:29.173+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:29.174 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:29.173+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:29.371 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:29.369+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:29.372 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:29.369+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:29.372 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:29.369+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:29.372 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:29.369+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:29.405 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:29.405+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:29.405 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:29.405+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:29.405 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:29.405+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:29.406 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:29.405+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:29.551 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:29.549+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:29.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:29.549+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:29.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:29.549+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:29.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:29.549+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:30.155 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:30.153+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:30.155 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:30.153+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:30.156 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:30.153+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:30.156 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:30.153+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:30.353 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:30.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:30.353 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:30.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:30.353 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:30.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:30.354 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:30.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:30.453 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:30.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:30.453 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:30.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:30.454 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:30.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:30.454 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:30.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:30.539 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:30.537+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:30.539 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:30.537+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:30.539 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:30.537+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:30.539 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:30.537+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:31.185 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:31.185+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:31.186 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:31.185+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:31.186 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:31.185+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:31.187 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:31.185+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:31.353 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:31.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:31.353 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:31.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:31.354 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:31.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:31.354 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:31.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:31.420 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:31.421+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:31.421 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:31.421+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:31.421 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:31.421+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:31.421 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:31.421+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:31.556 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:31.557+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:31.557 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:31.557+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:31.557 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:31.557+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:31.557 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:31.557+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:32.193 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:32.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:32.194 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:32.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:32.194 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:32.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:32.194 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:32.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:32.309 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:32.309+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:32.310 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:32.309+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:32.310 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:32.309+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:32.310 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:32.309+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:32.410 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:32.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:32.410 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:32.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:32.410 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:32.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:32.411 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:32.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:32.525 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:32.525+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:32.525 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:32.525+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:32.525 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:32.525+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:32.526 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:32.525+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:33.153 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:33.153+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:33.154 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:33.153+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:33.154 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:33.153+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:33.154 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:33.153+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:33.272 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:33.273+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:33.272 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:33.273+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:33.272 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:33.273+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:33.273 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:33.273+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:33.410 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:33.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:33.410 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:33.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:33.410 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:33.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:33.411 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:33.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:33.510 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:33.509+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:33.511 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:33.509+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:33.511 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:33.509+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:33.511 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:33.509+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:34.175 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:34.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:34.175 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:34.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:34.175 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:34.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:34.176 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:34.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:34.272 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:34.273+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:34.272 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:34.273+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:34.273 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:34.273+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:34.273 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:34.273+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:34.444 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:34.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:34.445 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:34.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:34.445 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:34.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:34.445 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:34.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:34.486 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:34.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:34.487 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:34.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:34.487 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:34.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:34.487 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:34.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:35.208 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:35.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:35.208 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:35.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:35.209 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:35.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:35.209 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:35.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:35.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:35.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:35.223 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:35.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:35.223 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:35.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:35.223 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:35.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:35.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:35.413+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:35.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:35.413+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:35.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:35.413+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:35.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:35.413+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:35.477 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:35.477+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:35.477 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:35.477+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:35.477 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:35.477+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:35.477 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:35.477+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:36.183 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:36.181+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:36.183 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:36.181+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:36.184 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:36.181+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:36.184 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:36.181+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:36.190 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:36.189+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:36.191 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:36.189+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:36.191 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:36.189+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:36.191 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:36.189+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:36.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:36.413+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:36.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:36.413+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:36.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:36.413+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:36.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:36.413+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:36.490 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:36.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:36.490 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:36.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:36.490 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:36.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:36.491 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:36.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:37.208 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:37.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:37.208 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:37.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:37.209 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:37.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:37.209 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:37.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:37.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:37.213+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:37.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:37.213+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:37.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:37.213+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:37.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:37.213+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:37.408 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:37.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:37.408 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:37.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:37.408 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:37.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:37.409 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:37.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:37.496 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:37.497+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:37.497 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:37.497+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:37.497 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:37.497+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:37.497 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:37.497+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:38.206 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:38.205+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:38.206 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:38.205+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:38.207 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:38.205+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:38.207 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:38.205+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:38.233 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:38.233+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:38.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:38.233+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:38.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:38.233+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:38.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:38.233+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:38.439 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:38.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:38.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:38.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:38.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:38.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:38.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:38.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:38.496 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:38.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:38.496 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:38.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:38.496 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:38.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:38.496 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:38.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:39.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:39.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:39.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:39.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:39.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:39.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:39.217 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:39.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:39.253 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:39.253+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:39.253 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:39.253+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:39.253 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:39.253+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:39.254 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:39.253+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:39.393 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:39.393+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:39.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:39.393+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:39.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:39.393+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:39.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:39.393+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:39.488 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:39.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:39.488 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:39.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:39.488 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:39.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:39.489 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:39.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:40.177 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:40.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:40.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:40.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:40.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:40.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:40.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:40.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:40.221 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:40.221+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:40.221 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:40.221+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:40.222 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:40.221+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:40.222 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:40.221+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:40.437 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:40.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:40.438 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:40.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:40.438 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:40.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:40.438 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:40.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:40.530 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:40.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:40.530 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:40.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:40.530 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:40.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:40.530 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:40.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:41.144 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:41.141+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:41.144 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:41.141+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:41.145 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:41.141+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:41.145 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:41.141+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:41.248 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:41.249+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:41.248 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:41.249+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:41.249 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:41.249+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:41.249 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:41.249+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:41.450 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:41.449+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:41.450 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:41.449+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:41.451 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:41.449+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:41.451 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:41.449+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:41.513 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:41.513+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:41.513 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:41.513+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:41.513 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:41.513+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:41.514 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:41.513+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:42.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:42.145+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:42.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:42.145+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:42.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:42.145+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:42.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:42.145+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:42.242 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:42.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:42.242 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:42.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:42.242 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:42.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:42.243 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:42.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:42.476 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:42.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:42.476 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:42.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:42.477 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:42.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:42.477 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:42.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:42.531 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:42.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:42.532 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:42.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:42.532 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:42.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:42.532 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:42.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:43.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:43.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:43.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:43.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:43.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:43.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:43.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:43.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:43.258 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:43.257+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:43.258 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:43.257+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:43.259 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:43.257+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:43.259 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:43.257+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:43.455 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:43.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:43.456 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:43.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:43.456 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:43.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:43.456 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:43.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:43.515 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:43.517+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:43.516 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:43.517+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:43.516 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:43.517+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:43.516 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:43.517+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:44.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:44.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:44.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:44.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:44.061 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:44.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:44.061 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:44.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:44.216 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:44.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:44.216 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:44.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:44.216 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:44.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:44.217 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:44.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:44.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:44.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:44.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:44.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:44.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:44.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:44.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:44.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:44.555 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:44.553+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:44.556 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:44.553+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:44.556 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:44.553+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:44.556 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:44.553+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:45.025 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:45.025+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:45.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:45.025+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:45.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:45.025+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:45.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:45.025+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:45.183 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:45.181+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:45.183 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:45.181+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:45.184 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:45.181+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:45.184 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:45.181+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:45.477 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:45.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:45.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:45.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:45.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:45.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:45.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:45.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:45.583 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:45.585+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:45.584 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:45.585+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:45.584 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:45.585+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:45.584 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:45.585+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:46.066 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:46.065+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:46.066 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:46.065+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:46.066 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:46.065+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:46.066 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:46.065+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:46.212 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:46.213+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:46.212 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:46.213+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:46.213 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:46.213+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:46.213 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:46.213+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:46.500 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:46.501+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:46.501 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:46.501+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:46.501 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:46.501+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:46.501 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:46.501+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:46.584 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:46.585+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:46.584 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:46.585+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:46.584 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:46.585+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:46.584 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:46.585+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:47.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:47.025+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:47.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:47.025+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:47.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:47.025+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:47.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:47.025+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:47.191 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:47.189+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:47.192 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:47.189+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:47.192 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:47.189+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:47.192 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:47.189+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:47.506 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:47.505+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:47.506 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:47.505+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:47.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:47.505+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:47.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:47.505+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:47.541 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:47.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:47.541 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:47.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:47.542 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:47.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:47.542 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:47.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:48.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:48.021+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:48.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:48.021+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:48.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:48.021+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:48.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:48.021+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:48.186 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:48.185+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:48.186 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:48.185+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:48.187 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:48.185+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:48.187 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:48.185+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:48.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:48.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:48.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:48.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:48.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:48.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:48.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:48.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:48.535 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:48.533+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:48.536 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:48.533+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:48.536 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:48.533+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:48.536 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:48.533+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:49.042 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:49.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:49.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:49.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:49.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:49.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:49.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:49.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:49.159 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:49.157+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:49.159 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:49.157+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:49.159 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:49.157+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:49.160 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:49.157+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:49.479 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:49.481+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:49.480 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:49.481+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:49.480 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:49.481+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:49.480 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:49.481+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:49.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:49.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:49.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:49.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:49.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:49.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:49.494 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:49.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:50.055 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:50.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:50.055 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:50.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:50.055 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:50.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:50.055 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:50.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:50.175 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:50.173+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:50.175 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:50.173+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:50.175 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:50.173+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:50.175 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:50.173+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:50.497 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:50.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:50.497 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:50.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:50.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:50.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:50.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:50.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:50.503 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:50.501+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:50.504 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:50.501+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:50.504 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:50.501+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:50.504 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:50.501+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:51.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:51.021+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:51.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:51.021+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:51.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:51.021+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:51.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:51.021+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:51.206 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:51.205+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:51.207 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:51.205+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:51.207 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:51.205+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:51.207 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:51.205+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:51.462 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:51.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:51.463 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:51.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:51.463 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:51.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:51.463 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:51.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:51.483 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:51.481+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:51.483 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:51.481+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:51.483 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:51.481+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:51.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:51.481+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:52.002 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:52.005+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:52.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:52.005+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:52.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:52.005+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:52.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:52.005+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:52.240 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:52.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:52.241 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:52.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:52.241 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:52.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:52.241 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:52.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:52.434 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:52.433+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:52.434 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:52.433+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:52.434 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:52.433+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:52.434 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:52.433+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:52.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:52.513+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:52.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:52.513+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:52.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:52.513+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:52.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:52.513+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:53.047 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:53.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:53.048 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:53.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:53.048 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:53.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:53.048 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:53.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:53.248 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:53.249+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:53.248 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:53.249+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:53.248 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:53.249+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:53.248 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:53.249+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:53.439 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:53.441+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:53.439 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:53.441+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:53.439 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:53.441+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:53.439 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:53.441+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:53.463 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:53.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:53.464 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:53.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:53.464 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:53.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:53.464 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:53.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:54.046 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:54.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:54.046 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:54.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:54.046 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:54.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:54.047 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:54.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:54.272 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:54.273+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:54.272 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:54.273+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:54.272 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:54.273+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:54.273 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:54.273+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:54.470 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:54.469+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:54.471 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:54.469+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:54.471 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:54.469+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:54.471 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:54.469+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:54.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:54.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:54.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:54.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:54.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:54.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:54.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:54.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:55.034 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:55.033+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:55.035 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:55.033+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:55.035 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:55.033+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:55.035 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:55.033+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:55.246 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:55.245+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:55.246 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:55.245+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:55.246 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:55.245+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:55.246 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:55.245+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:55.471 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:55.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:55.472 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:55.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:55.472 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:55.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:55.472 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:55.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:55.494 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:55.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:55.495 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:55.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:55.495 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:55.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:55.495 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:55.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:56.081 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:56.081+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:56.081 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:56.081+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:56.081 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:56.081+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:56.081 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:56.081+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:56.241 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:56.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:56.242 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:56.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:56.242 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:56.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:56.242 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:56.241+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:56.461 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:56.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:56.462 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:56.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:56.462 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:56.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:56.462 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:56.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:56.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:56.501+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:56.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:56.501+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:56.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:56.501+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:56.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:56.501+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:57.051 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:57.049+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:57.052 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:57.049+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:57.052 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:57.049+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:57.052 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:57.049+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:57.284 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:57.281+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:57.285 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:57.281+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:57.285 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:57.281+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:57.285 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:57.281+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:57.494 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:57.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:57.494 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:57.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:57.494 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:57.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:57.495 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:57.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:57.501 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:57.501+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:57.501 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:57.501+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:57.501 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:57.501+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:57.501 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:57.501+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:58.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:58.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:58.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:58.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:58.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:58.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:58.061 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:58.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:58.295 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:58.293+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:58.295 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:58.293+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:58.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:58.293+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:58.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:58.293+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:58.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:58.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:58.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:58.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:58.508 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:58.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:58.508 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:58.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:58.542 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:58.545+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:58.542 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:58.545+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:58.543 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:58.545+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:58.543 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:58.545+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:59.024 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:59.025+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:59.025 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:59.025+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:59.025 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:59.025+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:59.025 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:55:59.025+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:55:59.311 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:59.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:59.312 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:59.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:59.312 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:59.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:59.312 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:55:59.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:55:59.547 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:59.545+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:59.547 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:59.545+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:59.547 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:59.545+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:59.547 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:55:59.545+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:55:59.560 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:59.561+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:59.560 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:59.561+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:59.561 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:59.561+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:55:59.561 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:55:59.561+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:00.031 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:00.029+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:00.031 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:00.029+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:00.031 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:00.029+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:00.032 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:00.029+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:00.333 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:00.333+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:00.333 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:00.333+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:00.333 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:00.333+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:00.334 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:00.333+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:00.540 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:00.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:00.541 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:00.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:00.541 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:00.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:00.541 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:00.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:00.584 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:00.585+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:00.584 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:00.585+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:00.585 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:00.585+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:00.585 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:00.585+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:01.030 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:01.029+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:01.030 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:01.029+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:01.030 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:01.029+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:01.030 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:01.029+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:01.343 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:01.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:01.343 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:01.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:01.343 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:01.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:01.344 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:01.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:01.530 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:01.533+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:01.531 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:01.533+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:01.531 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:01.533+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:01.531 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:01.533+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:01.611 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:01.613+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:01.612 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:01.613+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:01.612 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:01.613+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:01.612 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:01.613+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:02.039 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:02.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:02.039 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:02.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:02.039 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:02.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:02.039 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:02.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:02.313 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:02.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:02.313 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:02.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:02.313 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:02.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:02.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:02.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:02.573 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:02.573+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:02.574 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:02.573+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:02.574 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:02.573+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:02.574 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:02.573+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:02.585 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:02.585+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:02.586 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:02.585+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:02.586 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:02.585+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:02.586 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:02.585+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:03.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:03.001+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:03.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:03.001+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:03.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:03.001+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:03.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:03.001+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:03.285 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:03.285+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:03.285 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:03.285+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:03.285 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:03.285+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:03.286 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:03.285+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:03.559 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:03.561+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:03.560 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:03.561+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:03.560 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:03.561+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:03.560 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:03.561+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:03.576 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:03.577+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:03.577 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:03.577+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:03.577 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:03.577+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:03.577 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:03.577+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:03.966 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:03.965+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:03.967 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:03.965+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:03.967 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:03.965+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:03.967 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:03.965+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:04.263 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:04.261+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:04.263 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:04.261+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:04.263 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:04.261+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:04.264 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:04.261+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:04.524 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:04.525+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:04.524 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:04.525+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:04.524 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:04.525+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:04.525 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:04.525+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:04.598 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:04.597+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:04.598 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:04.597+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:04.598 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:04.597+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:04.599 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:04.597+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:04.923 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:04.925+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:04.924 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:04.925+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:04.924 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:04.925+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:04.924 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:04.925+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:05.235 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:05.232+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:05.236 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:05.232+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:05.236 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:05.232+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:05.236 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:05.232+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:05.567 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:05.564+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:05.568 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:05.564+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:05.568 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:05.564+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:05.568 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:05.564+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:05.594 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:05.592+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:05.594 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:05.592+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:05.594 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:05.592+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:05.594 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:05.592+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:05.943 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:05.944+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:05.943 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:05.944+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:05.944 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:05.944+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:05.944 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:05.944+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:06.276 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:06.276+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:06.277 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:06.276+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:06.277 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:06.276+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:06.277 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:06.276+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:06.547 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:06.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:06.548 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:06.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:06.548 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:06.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:06.548 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:06.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:06.604 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:06.604+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:06.605 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:06.604+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:06.605 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:06.604+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:06.605 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:06.604+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:06.956 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:06.956+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:06.956 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:06.956+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:06.957 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:06.956+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:06.957 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:06.956+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:07.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:07.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:07.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:07.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:07.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:07.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:07.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:07.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:07.535 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:07.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:07.536 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:07.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:07.536 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:07.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:07.536 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:07.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:07.611 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:07.612+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:07.611 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:07.612+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:07.611 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:07.612+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:07.612 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:07.612+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:07.987 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:07.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:07.987 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:07.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:07.987 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:07.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:07.987 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:07.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:08.327 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:08.324+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:08.327 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:08.324+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:08.328 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:08.324+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:08.328 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:08.324+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:08.534 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:08.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:08.534 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:08.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:08.534 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:08.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:08.535 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:08.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:08.596 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:08.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:08.596 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:08.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:08.596 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:08.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:08.596 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:08.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:08.951 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:08.952+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:08.951 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:08.952+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:08.952 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:08.952+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:08.952 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:08.952+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:09.316 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:09.316+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:09.316 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:09.316+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:09.316 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:09.316+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:09.316 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:09.316+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:09.503 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:09.504+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:09.503 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:09.504+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:09.504 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:09.504+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:09.504 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:09.504+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:09.555 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:09.552+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:09.555 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:09.552+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:09.555 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:09.552+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:09.556 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:09.552+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:09.950 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:09.948+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:09.950 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:09.948+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:09.951 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:09.948+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:09.951 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:09.948+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:10.317 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:10.316+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:10.318 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:10.316+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:10.318 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:10.316+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:10.318 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:10.316+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:10.469 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:10.468+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:10.470 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:10.468+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:10.470 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:10.468+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:10.470 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:10.468+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:10.519 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:10.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:10.519 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:10.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:10.519 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:10.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:10.519 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:10.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:10.981 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:10.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:10.981 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:10.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:10.981 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:10.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:10.981 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:10.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:11.343 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:11.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:11.343 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:11.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:11.344 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:11.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:11.344 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:11.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:11.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:11.512+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:11.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:11.512+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:11.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:11.512+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:11.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:11.512+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:11.536 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:11.536+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:11.536 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:11.536+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:11.536 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:11.536+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:11.537 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:11.536+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:11.985 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:11.984+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:11.985 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:11.984+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:11.985 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:11.984+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:11.986 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:11.984+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:12.332 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:12.332+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:12.332 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:12.332+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:12.332 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:12.332+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:12.333 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:12.332+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:12.475 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:12.472+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:12.475 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:12.472+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:12.476 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:12.472+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:12.476 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:12.472+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:12.488 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:12.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:12.488 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:12.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:12.488 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:12.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:12.489 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:12.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:13.009 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:13.008+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:13.009 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:13.008+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:13.010 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:13.008+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:13.010 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:13.008+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:13.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:13.304+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:13.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:13.304+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:13.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:13.304+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:13.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:13.304+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:13.441 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:13.440+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:13.441 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:13.440+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:13.441 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:13.440+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:13.442 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:13.440+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:13.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:13.484+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:13.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:13.484+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:13.488 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:13.484+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:13.488 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:13.484+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:13.994 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:13.992+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:13.995 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:13.992+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:13.995 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:13.992+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:13.995 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:13.992+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:14.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:14.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:14.342 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:14.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:14.342 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:14.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:14.342 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:14.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:14.419 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:14.420+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:14.419 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:14.420+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:14.420 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:14.420+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:14.420 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:14.420+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:14.452 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:14.452+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:14.452 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:14.452+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:14.452 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:14.452+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:14.453 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:14.452+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:14.974 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:14.972+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:14.975 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:14.972+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:14.975 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:14.972+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:14.975 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:14.972+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:15.313 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:15.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:15.313 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:15.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:15.313 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:15.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:15.313 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:15.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:15.458 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:15.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:15.459 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:15.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:15.459 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:15.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:15.459 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:15.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:15.460 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:15.456+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:15.460 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:15.456+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:15.460 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:15.456+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:15.461 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:15.456+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:15.950 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:15.948+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:15.950 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:15.948+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:15.950 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:15.948+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:15.950 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:15.948+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:16.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:16.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:16.361 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:16.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:16.361 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:16.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:16.361 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:16.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:16.462 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:16.460+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:16.462 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:16.460+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:16.462 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:16.460+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:16.462 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:16.460+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:16.490 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:16.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:16.490 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:16.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:16.491 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:16.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:16.491 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:16.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:16.981 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:16.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:16.981 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:16.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:16.982 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:16.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:16.982 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:16.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:17.392 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:17.392+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:17.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:17.392+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:17.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:17.392+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:17.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:17.392+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:17.501 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:17.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:17.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:17.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:17.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:17.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:17.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:17.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:17.511 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:17.512+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:17.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:17.512+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:17.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:17.512+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:17.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:17.512+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:18.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:18.020+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:18.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:18.020+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:18.022 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:18.020+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:18.022 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:18.020+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:18.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:18.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:18.353 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:18.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:18.353 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:18.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:18.353 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:18.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:18.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:18.464+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:18.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:18.464+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:18.466 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:18.464+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:18.466 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:18.464+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:18.540 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:18.540+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:18.540 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:18.540+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:18.540 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:18.540+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:18.540 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:18.540+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:19.012 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:19.012+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:19.013 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:19.012+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:19.013 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:19.012+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:19.013 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:19.012+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:19.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:19.372+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:19.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:19.372+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:19.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:19.372+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:19.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:19.372+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:19.500 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:19.500+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:19.500 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:19.500+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:19.501 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:19.500+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:19.501 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:19.500+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:19.567 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:19.568+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:19.568 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:19.568+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:19.568 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:19.568+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:19.568 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:19.568+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:20.027 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:20.028+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:20.028 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:20.028+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:20.028 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:20.028+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:20.028 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:20.028+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:20.366 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:20.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:20.366 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:20.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:20.367 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:20.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:20.367 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:20.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:20.542 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:20.544+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:20.542 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:20.544+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:20.543 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:20.544+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:20.543 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:20.544+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:20.549 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:20.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:20.549 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:20.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:20.549 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:20.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:20.549 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:20.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:21.045 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:21.044+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:21.045 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:21.044+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:21.045 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:21.044+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:21.046 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:21.044+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:21.377 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:21.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:21.378 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:21.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:21.378 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:21.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:21.378 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:21.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:21.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:21.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:21.508 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:21.508+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:21.508 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:21.508+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:21.508 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:21.508+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:21.556 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:21.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:21.557 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:21.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:21.557 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:21.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:21.557 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:21.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:22.068 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:22.068+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:22.068 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:22.068+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:22.069 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:22.068+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:22.069 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:22.068+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:22.390 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:22.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:22.390 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:22.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:22.391 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:22.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:22.391 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:22.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:22.489 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:22.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:22.489 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:22.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:22.489 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:22.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:22.489 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:22.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:22.522 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:22.520+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:22.523 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:22.520+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:22.523 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:22.520+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:22.523 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:22.520+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:23.025 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:23.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:23.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:23.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:23.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:23.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:23.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:23.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:23.367 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:23.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:23.367 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:23.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:23.367 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:23.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:23.367 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:23.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:23.490 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:23.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:23.490 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:23.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:23.491 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:23.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:23.491 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:23.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:23.529 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:23.528+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:23.529 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:23.528+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:23.530 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:23.528+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:23.530 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:23.528+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:24.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:24.020+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:24.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:24.020+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:24.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:24.020+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:24.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:24.020+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:24.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:24.368+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:24.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:24.368+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:24.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:24.368+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:24.371 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:24.368+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:24.510 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:24.508+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:24.511 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:24.508+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:24.511 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:24.508+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:24.511 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:24.508+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:24.524 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:24.524+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:24.525 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:24.524+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:24.525 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:24.524+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:24.525 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:24.524+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:25.006 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:25.004+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:25.006 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:25.004+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:25.006 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:25.004+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:25.006 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:25.004+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:25.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:25.372+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:25.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:25.372+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:25.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:25.372+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:25.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:25.372+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:25.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:25.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:25.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:25.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:25.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:25.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:25.504 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:25.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:25.508 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:25.508+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:25.508 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:25.508+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:25.508 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:25.508+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:25.508 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:25.508+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:25.984 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:25.984+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:25.985 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:25.984+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:25.985 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:25.984+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:25.985 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:25.984+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:26.347 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:26.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:26.347 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:26.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:26.348 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:26.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:26.348 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:26.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:26.486 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:26.484+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:26.486 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:26.484+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:26.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:26.484+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:26.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:26.484+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:26.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:26.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:26.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:26.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:26.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:26.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:26.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:26.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:26.966 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:26.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:26.967 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:26.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:26.967 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:26.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:26.967 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:26.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:27.388 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:27.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:27.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:27.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:27.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:27.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:27.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:27.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:27.473 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:27.472+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:27.473 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:27.472+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:27.473 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:27.472+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:27.474 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:27.472+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:27.490 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:27.492+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:27.491 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:27.492+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:27.491 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:27.492+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:27.491 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:27.492+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:27.989 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:27.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:27.990 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:27.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:27.990 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:27.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:27.990 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:27.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:28.380 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:28.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:28.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:28.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:28.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:28.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:28.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:28.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:28.463 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:28.460+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:28.463 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:28.460+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:28.463 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:28.460+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:28.464 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:28.460+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:28.506 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:28.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:28.506 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:28.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:28.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:28.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:28.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:28.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:28.967 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:28.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:28.967 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:28.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:28.968 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:28.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:28.968 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:28.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:29.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:29.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:29.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:29.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:29.339 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:29.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:29.339 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:29.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:29.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:29.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:29.494 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:29.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:29.494 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:29.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:29.494 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:29.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:29.550 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:29.552+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:29.551 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:29.552+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:29.551 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:29.552+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:29.551 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:29.552+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:29.991 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:29.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:29.992 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:29.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:29.992 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:29.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:29.992 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:29.992+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:30.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:30.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:30.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:30.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:30.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:30.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:30.361 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:30.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:30.473 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:30.472+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:30.473 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:30.472+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:30.474 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:30.472+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:30.474 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:30.472+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:30.515 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:30.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:30.516 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:30.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:30.516 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:30.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:30.516 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:30.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:30.977 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:30.976+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:30.978 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:30.976+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:30.978 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:30.976+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:30.978 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:30.976+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:31.345 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:31.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:31.345 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:31.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:31.345 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:31.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:31.346 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:31.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:31.489 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:31.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:31.490 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:31.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:31.490 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:31.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:31.490 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:31.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:31.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:31.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:31.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:31.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:31.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:31.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:31.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:31.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:32.022 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:32.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:32.023 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:32.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:32.023 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:32.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:32.023 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:32.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:32.306 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:32.304+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:32.306 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:32.304+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:32.306 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:32.304+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:32.306 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:32.304+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:32.457 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:32.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:32.457 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:32.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:32.458 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:32.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:32.458 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:32.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:32.533 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:32.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:32.533 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:32.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:32.533 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:32.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:32.533 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:32.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:32.990 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:32.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:32.991 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:32.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:32.991 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:32.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:32.991 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:32.988+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:33.290 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:33.288+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:33.290 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:33.288+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:33.290 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:33.288+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:33.291 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:33.288+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:33.496 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:33.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:33.496 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:33.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:33.497 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:33.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:33.497 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:33.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:33.509 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:33.508+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:33.509 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:33.508+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:33.509 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:33.508+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:33.510 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:33.508+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:34.005 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:34.004+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:34.006 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:34.004+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:34.006 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:34.004+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:34.006 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:34.004+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:34.321 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:34.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:34.322 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:34.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:34.322 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:34.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:34.322 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:34.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:34.492 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:34.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:34.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:34.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:34.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:34.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:34.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:34.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:34.505 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:34.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:34.505 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:34.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:34.506 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:34.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:34.506 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:34.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:34.980 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:34.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:34.980 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:34.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:34.980 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:34.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:34.981 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:34.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:35.295 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:35.292+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:35.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:35.296+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:35.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:35.296+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:35.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:35.296+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:35.458 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:35.460+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:35.458 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:35.460+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:35.459 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:35.460+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:35.459 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:35.460+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:35.501 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:35.500+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:35.501 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:35.500+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:35.502 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:35.500+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:35.502 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:35.500+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:35.999 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:35.996+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:35.999 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:35.996+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:36.000 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:35.996+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:36.000 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:35.996+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:36.324 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:36.324+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:36.324 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:36.324+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:36.324 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:36.324+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:36.325 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:36.324+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:36.410 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:36.408+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:36.411 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:36.408+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:36.411 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:36.408+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:36.411 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:36.408+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:36.522 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:36.524+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:36.522 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:36.524+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:36.522 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:36.524+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:36.523 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:36.524+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:36.992 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:36.992+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:36.993 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:36.992+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:36.993 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:36.992+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:36.993 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:36.992+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:37.302 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:37.300+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:37.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:37.300+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:37.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:37.300+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:37.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:37.300+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:37.383 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:37.384+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:37.383 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:37.384+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:37.383 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:37.384+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:37.383 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:37.384+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:37.535 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:37.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:37.536 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:37.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:37.536 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:37.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:37.536 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:37.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:38.023 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:38.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:38.024 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:38.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:38.024 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:38.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:38.024 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:38.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:38.321 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:56:38.320+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:56:38.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:38.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:38.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:38.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:38.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:38.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:38.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:38.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:38.342 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:38.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:38.343 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:38.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:38.343 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:38.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:38.343 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:38.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:38.513 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:38.512+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:38.514 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:38.512+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:38.514 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:38.512+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:38.514 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:38.512+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:38.991 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:38.992+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:38.992 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:38.992+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:38.992 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:38.992+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:38.992 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:38.992+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:39.295 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:39.296+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:39.295 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:39.296+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:39.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:39.296+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:39.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:39.296+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:39.391 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:39.392+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:39.392 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:39.392+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:39.392 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:39.392+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:39.392 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:39.392+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:39.533 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:39.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:39.533 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:39.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:39.533 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:39.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:39.533 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:39.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:40.014 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:40.012+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:40.014 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:40.012+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:40.014 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:40.012+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:40.015 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:40.012+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:40.291 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:40.292+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:40.291 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:40.292+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:40.291 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:40.292+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:40.292 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:40.292+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:40.393 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:40.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:40.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:40.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:40.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:40.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:40.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:40.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:40.487 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:40.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:40.488 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:40.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:40.488 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:40.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:40.488 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:40.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:40.998 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:40.996+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:40.998 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:40.996+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:40.998 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:40.996+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:40.998 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:40.996+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:41.305 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:41.304+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:41.306 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:41.304+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:41.306 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:41.304+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:41.306 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:41.304+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:41.356 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:41.356+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:41.357 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:41.356+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:41.357 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:41.356+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:41.357 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:41.356+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:41.518 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:41.516+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:41.519 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:41.516+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:41.519 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:41.516+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:41.519 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:41.516+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:42.039 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:42.036+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:42.039 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:42.036+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:42.039 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:42.036+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:42.040 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:42.036+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:42.334 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:42.332+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:42.334 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:42.332+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:42.335 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:42.332+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:42.335 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:42.332+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:42.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:42.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:42.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:42.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:42.373 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:42.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:42.373 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:42.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:42.501 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:42.500+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:42.501 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:42.500+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:42.501 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:42.500+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:42.502 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:42.500+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:43.042 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:43.044+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:43.042 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:43.044+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:43.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:43.044+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:43.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:43.044+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:43.321 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:56:43.320+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:56:43.345 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:43.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:43.346 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:43.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:43.346 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:43.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:43.346 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:43.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:43.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:43.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:43.380 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:43.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:43.380 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:43.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:43.380 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:43.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:43.458 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:43.460+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:43.459 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:43.460+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:43.459 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:43.460+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:43.459 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:43.460+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:44.074 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:44.072+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:44.075 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:44.072+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:44.075 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:44.072+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:44.075 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:44.072+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:44.345 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:44.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:44.346 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:44.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:44.346 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:44.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:44.346 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:44.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:44.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:44.388+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:44.390 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:44.388+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:44.390 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:44.388+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:44.391 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:44.388+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:44.486 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:44.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:44.487 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:44.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:44.487 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:44.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:44.487 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:44.488+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:45.025 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:45.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:45.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:45.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:45.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:45.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:45.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:45.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:45.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:45.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:45.351 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:45.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:45.351 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:45.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:45.351 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:45.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:45.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:45.412+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:45.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:45.412+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:45.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:45.412+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:45.414 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:45.412+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:45.440 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:45.440+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:45.440 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:45.440+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:45.441 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:45.440+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:45.441 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:45.440+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:45.987 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:45.984+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:45.987 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:45.984+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:45.987 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:45.984+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:45.988 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:45.984+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:46.365 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:46.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:46.366 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:46.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:46.366 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:46.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:46.366 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:46.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:46.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:46.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:46.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:46.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:46.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:46.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:46.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:46.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:46.457 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:46.456+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:46.457 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:46.456+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:46.457 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:46.456+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:46.457 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:46.456+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:46.965 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:46.964+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:46.965 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:46.964+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:46.965 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:46.964+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:46.965 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:46.964+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:47.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:47.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:47.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:47.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:47.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:47.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:47.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:47.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:47.405 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:47.404+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:47.406 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:47.404+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:47.406 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:47.404+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:47.407 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:47.404+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:47.424 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:47.424+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:47.425 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:47.424+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:47.425 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:47.424+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:47.426 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:47.424+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:47.999 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:48.000+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:48.000 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:48.000+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:48.000 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:48.000+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:48.000 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:48.000+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:48.323 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:56:48.320+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:56:48.351 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:48.348+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:48.351 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:48.348+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:48.351 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:48.348+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:48.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:48.348+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:48.396 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:48.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:48.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:48.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:48.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:48.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:48.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:48.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:48.431 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:48.428+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:48.431 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:48.428+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:48.431 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:48.428+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:48.431 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:48.428+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:48.968 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:48.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:48.968 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:48.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:48.968 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:48.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:48.968 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:48.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:49.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:49.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:49.388 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:49.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:49.388 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:49.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:49.388 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:49.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:49.442 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:49.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:49.442 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:49.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:49.442 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:49.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:49.442 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:49.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:49.453 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:49.452+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:49.454 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:49.452+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:49.454 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:49.452+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:49.454 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:49.452+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:49.963 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:49.964+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:49.963 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:49.964+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:49.964 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:49.964+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:49.964 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:49.964+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:50.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:50.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:50.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:50.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:50.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:50.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:50.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:50.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:50.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:50.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:50.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:50.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:50.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:50.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:50.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:50.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:50.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:50.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:50.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:50.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:50.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:50.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:50.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:50.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:50.954 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:50.952+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:50.955 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:50.952+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:50.955 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:50.952+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:50.955 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:50.952+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:51.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:51.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:51.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:51.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:51.363 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:51.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:51.363 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:51.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:51.444 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:51.444+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:51.444 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:51.444+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:51.444 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:51.444+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:51.445 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:51.444+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:51.460 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:51.460+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:51.460 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:51.460+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:51.461 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:51.460+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:51.461 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:51.460+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:51.910 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:51.912+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:51.910 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:51.912+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:51.910 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:51.912+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:51.911 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:51.912+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:52.398 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:52.396+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:52.398 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:52.396+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:52.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:52.396+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:52.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:52.396+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:52.456 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:52.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:52.456 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:52.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:52.457 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:52.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:52.457 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:52.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:52.511 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:52.508+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:52.511 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:52.508+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:52.511 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:52.508+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:52.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:52.508+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:52.872 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:52.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:52.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:52.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:52.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:52.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:52.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:52.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:53.323 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:56:53.324+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:56:53.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:53.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:53.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:53.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:53.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:53.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:53.384 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:53.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:53.461 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:53.460+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:53.461 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:53.460+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:53.461 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:53.460+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:53.462 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:53.460+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:53.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:53.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:53.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:53.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:53.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:53.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:53.553 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:53.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:53.894 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:53.896+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:53.895 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:53.896+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:53.895 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:53.896+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:53.895 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:53.896+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:54.349 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:54.348+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:54.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:54.348+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:54.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:54.348+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:54.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:54.348+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:54.434 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:54.436+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:54.434 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:54.436+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:54.434 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:54.436+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:54.435 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:54.436+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:54.540 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:54.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:54.540 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:54.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:54.541 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:54.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:54.541 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:54.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:54.855 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:54.852+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:54.855 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:54.852+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:54.856 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:54.852+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:54.856 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:54.852+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:55.340 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:55.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:55.340 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:55.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:55.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:55.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:55.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:55.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:55.469 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:55.472+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:55.470 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:55.472+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:55.470 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:55.472+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:55.470 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:55.472+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:55.579 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:55.576+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:55.579 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:55.576+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:55.580 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:55.576+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:55.580 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:55.576+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:55.843 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:55.844+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:55.843 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:55.844+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:55.843 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:55.844+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:55.844 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:55.844+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:56.384 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:56.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:56.384 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:56.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:56.384 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:56.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:56.385 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:56.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:56.457 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:56.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:56.457 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:56.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:56.458 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:56.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:56.458 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:56.456+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:56.557 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:56.560+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:56.558 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:56.560+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:56.558 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:56.560+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:56.558 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:56.560+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:56.868 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:56.868+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:56.868 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:56.868+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:56.869 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:56.868+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:56.869 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:56.868+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:57.413 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:57.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:57.413 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:57.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:57.414 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:57.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:57.414 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:57.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:57.426 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:57.428+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:57.426 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:57.428+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:57.426 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:57.428+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:57.427 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:57.428+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:57.601 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:57.604+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:57.602 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:57.604+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:57.602 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:57.604+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:57.602 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:57.604+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:57.854 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:57.856+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:57.854 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:57.856+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:57.854 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:57.856+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:57.855 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:57.856+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:58.324 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:56:58.324+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 34 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:56:58.376 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:58.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:58.376 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:58.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:58.377 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:58.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:58.377 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:58.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:58.453 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:58.452+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:58.454 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:58.452+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:58.454 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:58.452+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:58.454 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:58.452+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:58.606 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:58.608+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:58.606 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:58.608+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:58.606 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:58.608+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:58.606 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:58.608+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:58.884 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:58.884+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:58.885 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:58.884+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:58.885 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:58.884+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:58.885 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:58.884+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:59.409 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:59.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:59.410 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:59.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:59.410 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:59.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:59.410 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:56:59.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:56:59.495 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:59.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:59.496 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:59.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:59.496 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:59.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:59.496 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:56:59.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:56:59.616 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:59.616+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:59.616 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:59.616+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:59.617 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:59.616+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:59.617 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:56:59.616+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:56:59.879 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:59.880+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:59.880 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:59.880+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:59.880 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:59.880+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:56:59.880 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:56:59.880+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:00.403 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:00.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:00.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:00.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:00.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:00.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:00.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:00.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:00.497 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:00.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:00.497 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:00.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:00.497 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:00.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:00.497 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:00.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:00.621 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:00.620+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:00.621 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:00.620+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:00.621 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:00.620+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:00.622 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:00.620+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:00.841 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:00.840+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:00.841 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:00.840+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:00.841 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:00.840+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:00.841 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:00.840+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:01.356 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:01.356+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:01.356 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:01.356+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:01.356 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:01.356+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:01.356 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:01.356+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:01.481 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:01.480+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:01.481 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:01.480+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:01.482 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:01.480+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:01.482 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:01.480+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:01.667 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:01.668+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:01.668 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:01.668+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:01.668 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:01.668+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:01.668 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:01.668+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:01.879 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:01.880+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:01.879 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:01.880+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:01.880 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:01.880+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:01.880 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:01.880+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:02.402 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:02.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:02.402 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:02.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:02.402 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:02.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:02.402 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:02.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:02.444 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:02.444+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:02.445 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:02.444+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:02.445 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:02.444+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:02.445 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:02.444+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:02.686 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:02.684+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:02.687 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:02.684+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:02.687 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:02.684+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:02.687 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:02.684+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:02.891 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:02.892+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:02.892 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:02.892+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:02.892 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:02.892+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:02.892 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:02.892+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:03.325 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:57:03.324+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 62 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:57:03.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:03.400+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:03.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:03.400+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:03.398 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:03.400+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:03.398 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:03.400+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:03.398 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:03.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:03.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:03.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:03.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:03.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:03.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:03.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:03.676 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:03.676+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:03.676 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:03.676+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:03.677 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:03.676+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:03.677 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:03.676+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:03.872 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:03.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:03.872 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:03.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:03.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:03.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:03.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:03.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:04.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:04.412+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:04.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:04.412+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:04.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:04.412+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:04.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:04.412+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:04.421 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:04.420+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:04.421 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:04.420+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:04.421 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:04.420+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:04.421 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:04.420+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:04.663 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:04.664+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:04.663 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:04.664+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:04.663 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:04.664+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:04.663 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:04.664+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:04.877 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:04.876+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:04.878 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:04.876+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:04.878 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:04.876+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:04.878 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:04.876+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:05.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:05.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:05.384 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:05.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:05.384 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:05.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:05.384 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:05.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:05.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:05.392+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:05.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:05.392+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:05.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:05.392+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:05.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:05.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:05.669 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:05.672+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:05.670 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:05.672+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:05.670 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:05.672+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:05.670 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:05.672+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:05.870 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:05.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:05.871 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:05.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:05.871 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:05.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:05.871 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:05.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:06.414 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:06.416+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:06.414 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:06.416+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:06.414 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:06.416+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:06.415 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:06.416+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:06.432 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:06.432+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:06.433 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:06.432+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:06.433 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:06.432+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:06.433 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:06.432+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:06.689 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:06.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:06.689 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:06.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:06.690 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:06.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:06.690 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:06.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:06.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:06.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:06.874 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:06.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:06.874 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:06.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:06.874 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:06.872+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:07.411 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:07.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:07.411 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:07.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:07.411 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:07.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:07.412 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:07.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:07.428 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:07.428+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:07.428 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:07.428+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:07.428 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:07.428+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:07.428 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:07.428+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:07.738 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:07.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:07.738 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:07.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:07.739 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:07.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:07.739 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:07.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:07.854 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:07.852+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:07.855 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:07.852+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:07.855 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:07.852+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:07.855 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:07.852+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:08.325 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:57:08.324+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 91 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:57:08.390 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:08.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:08.390 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:08.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:08.390 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:08.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:08.390 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:08.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:08.422 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:08.420+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:08.423 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:08.420+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:08.423 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:08.420+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:08.423 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:08.420+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:08.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:08.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:08.724 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:08.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:08.724 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:08.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:08.724 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:08.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:08.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:08.864+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:08.862 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:08.864+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:08.862 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:08.864+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:08.862 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:08.864+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:09.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:09.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:09.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:09.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:09.353 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:09.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:09.353 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:09.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:09.471 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:09.472+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:09.471 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:09.472+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:09.472 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:09.472+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:09.472 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:09.472+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:09.752 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:09.752+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:09.753 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:09.752+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:09.753 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:09.752+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:09.753 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:09.752+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:09.869 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:09.868+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:09.869 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:09.868+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:09.870 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:09.868+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:09.870 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:09.868+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:10.330 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:10.328+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:10.330 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:10.328+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:10.331 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:10.328+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:10.331 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:10.328+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:10.422 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:10.424+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:10.423 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:10.424+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:10.423 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:10.424+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:10.423 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:10.424+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:10.707 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:10.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:10.707 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:10.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:10.707 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:10.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:10.707 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:10.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:10.919 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:10.920+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:10.919 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:10.920+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:10.919 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:10.920+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:10.920 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:10.920+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:11.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:11.356+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:11.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:11.356+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:11.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:11.356+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:11.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:11.356+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:11.417 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:11.416+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:11.417 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:11.416+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:11.417 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:11.416+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:11.418 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:11.416+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:11.725 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:11.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:11.726 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:11.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:11.726 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:11.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:11.726 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:11.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:11.880 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:11.879+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:11.880 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:11.879+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:11.880 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:11.879+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:11.880 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:11.879+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:12.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:12.359+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:12.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:12.359+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:12.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:12.359+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:12.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:12.359+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:12.445 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:12.443+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:12.445 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:12.443+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:12.445 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:12.443+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:12.445 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:12.443+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:12.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:12.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:12.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:12.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:12.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:12.759+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:12.760 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:12.759+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:12.841 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:12.839+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:12.842 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:12.839+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:12.842 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:12.839+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:12.842 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:12.839+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:13.327 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:57:13.323+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 121 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:57:13.343 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:13.339+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:13.343 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:13.339+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:13.343 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:13.339+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:13.343 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:13.339+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:13.420 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:13.419+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:13.420 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:13.419+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:13.420 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:13.419+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:13.420 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:13.419+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:13.726 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:13.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:13.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:13.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:13.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:13.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:13.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:13.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:13.828 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:13.827+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:13.828 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:13.827+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:13.828 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:13.827+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:13.828 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:13.827+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:14.380 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:14.379+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:14.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:14.379+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:14.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:14.379+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:14.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:14.379+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:14.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:14.399+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:14.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:14.399+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:14.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:14.399+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:14.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:14.399+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:14.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:14.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:14.728 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:14.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:14.728 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:14.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:14.728 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:14.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:14.837 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:14.835+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:14.838 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:14.835+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:14.838 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:14.835+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:14.838 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:14.835+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:15.361 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:15.363+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:15.361 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:15.363+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:15.361 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:15.363+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:15.362 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:15.363+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:15.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:15.379+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:15.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:15.379+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:15.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:15.379+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:15.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:15.379+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:15.712 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:15.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:15.713 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:15.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:15.713 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:15.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:15.713 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:15.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:15.850 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:15.851+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:15.851 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:15.851+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:15.851 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:15.851+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:15.851 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:15.851+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:16.369 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:16.367+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:16.369 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:16.367+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:16.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:16.367+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:16.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:16.367+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:16.407 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:16.407+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:16.408 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:16.407+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:16.408 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:16.407+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:16.408 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:16.407+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:16.666 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:16.663+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:16.667 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:16.663+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:16.667 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:16.663+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:16.667 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:16.663+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:16.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:16.863+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:16.862 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:16.863+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:16.862 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:16.863+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:16.862 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:16.863+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:17.339 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:17.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:17.339 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:17.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:17.339 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:17.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:17.340 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:17.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:17.431 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:17.431+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:17.432 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:17.431+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:17.432 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:17.431+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:17.432 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:17.431+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:17.704 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:17.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:17.705 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:17.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:17.705 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:17.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:17.705 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:17.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:17.884 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:17.883+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:17.885 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:17.883+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:17.885 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:17.883+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:17.885 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:17.883+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:18.326 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:18.327+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:18.327 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:18.327+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:18.327 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:18.327+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:18.327 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:18.327+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:18.328 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:57:18.327+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 151 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:57:18.410 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:18.411+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:18.410 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:18.411+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:18.411 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:18.411+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:18.411 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:18.411+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:18.691 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:18.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:18.691 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:18.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:18.692 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:18.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:18.692 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:18.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:18.906 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:18.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:18.907 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:18.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:18.907 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:18.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:18.907 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:18.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:19.280 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:19.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:19.280 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:19.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:19.280 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:19.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:19.280 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:19.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:19.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:19.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:19.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:19.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:19.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:19.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:19.396 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:19.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:19.707 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:19.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:19.707 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:19.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:19.708 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:19.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:19.708 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:19.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:19.944 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:19.943+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:19.945 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:19.943+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:19.945 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:19.943+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:19.945 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:19.943+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:20.252 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:20.251+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:20.252 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:20.251+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:20.252 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:20.251+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:20.252 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:20.251+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:20.358 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:20.355+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:20.358 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:20.355+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:20.358 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:20.355+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:20.358 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:20.355+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:20.722 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:20.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:20.722 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:20.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:20.722 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:20.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:20.722 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:20.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:20.950 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:20.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:20.951 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:20.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:20.951 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:20.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:20.951 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:20.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:21.224 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:21.223+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:21.224 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:21.223+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:21.225 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:21.223+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:21.225 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:21.223+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:21.391 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:21.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:21.391 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:21.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:21.391 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:21.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:21.392 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:21.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:21.769 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:21.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:21.770 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:21.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:21.770 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:21.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:21.770 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:21.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:21.939 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:21.939+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:21.940 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:21.939+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:21.940 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:21.939+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:21.940 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:21.939+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:22.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:22.199+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:22.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:22.199+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:22.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:22.199+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:22.201 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:22.199+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:22.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:22.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:22.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:22.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:22.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:22.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:22.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:22.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:22.766 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:22.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:22.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:22.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:22.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:22.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:22.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:22.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:22.943 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:22.943+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:22.944 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:22.943+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:22.944 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:22.943+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:22.944 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:22.943+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:23.195 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:23.195+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:23.195 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:23.195+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:23.196 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:23.195+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:23.196 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:23.195+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:23.328 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:57:23.327+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 181 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:57:23.359 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:23.359+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:23.359 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:23.359+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:23.359 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:23.359+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:23.360 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:23.359+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:23.809 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:23.811+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:23.809 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:23.811+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:23.809 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:23.811+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:23.810 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:23.811+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:23.905 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:23.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:23.905 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:23.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:23.905 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:23.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:23.905 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:23.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:24.232 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:24.231+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:24.232 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:24.231+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:24.233 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:24.231+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:24.233 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:24.231+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:24.330 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:24.331+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:24.331 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:24.331+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:24.331 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:24.331+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:24.331 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:24.331+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:24.802 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:24.799+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:24.803 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:24.799+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:24.803 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:24.799+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:24.803 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:24.799+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:24.935 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:24.935+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:24.936 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:24.935+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:24.936 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:24.935+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:24.936 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:24.935+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:25.192 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:25.191+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:25.192 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:25.191+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:25.193 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:25.191+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:25.193 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:25.191+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:25.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:25.287+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:25.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:25.287+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:25.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:25.287+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:25.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:25.287+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:25.781 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:25.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:25.782 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:25.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:25.782 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:25.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:25.782 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:25.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:25.975 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:25.975+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:25.975 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:25.975+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:25.976 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:25.975+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:25.976 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:25.975+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:26.217 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:26.219+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:26.217 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:26.219+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:26.218 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:26.219+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:26.218 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:26.219+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:26.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:26.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:26.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:26.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:26.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:26.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:26.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:26.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:26.758 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:26.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:26.758 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:26.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:26.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:26.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:26.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:26.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:26.993 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:26.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:26.993 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:26.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:26.994 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:26.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:26.994 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:26.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:27.182 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:27.179+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:27.183 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:27.179+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:27.183 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:27.179+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:27.183 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:27.179+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:27.287 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:27.287+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:27.288 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:27.287+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:27.288 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:27.287+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:27.288 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:27.287+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:27.763 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:27.763+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:27.763 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:27.763+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:27.764 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:27.763+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:27.764 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:27.763+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:27.983 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:27.983+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:27.983 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:27.983+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:27.983 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:27.983+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:27.984 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:27.983+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:28.232 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:28.231+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:28.232 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:28.231+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:28.233 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:28.231+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:28.233 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:28.231+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:28.288 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:28.287+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:28.288 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:28.287+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:28.288 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:28.287+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:28.289 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:28.287+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:28.329 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:57:28.327+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 210 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:57:28.724 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:28.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:28.725 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:28.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:28.725 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:28.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:28.725 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:28.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:28.998 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:28.999+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:28.999 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:28.999+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:28.999 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:28.999+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:28.999 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:28.999+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:29.264 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:29.263+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:29.264 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:29.263+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:29.264 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:29.263+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:29.265 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:29.263+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:29.284 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:29.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:29.284 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:29.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:29.285 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:29.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:29.285 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:29.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:29.677 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:29.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:29.677 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:29.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:29.677 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:29.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:29.678 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:29.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:30.017 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:30.015+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:30.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:30.015+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:30.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:30.015+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:30.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:30.015+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:30.239 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:30.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:30.239 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:30.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:30.240 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:30.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:30.240 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:30.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:30.242 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:30.239+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:30.242 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:30.239+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:30.243 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:30.239+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:30.243 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:30.239+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:30.722 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:30.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:30.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:30.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:30.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:30.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:30.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:30.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:31.046 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:31.043+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:31.047 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:31.043+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:31.047 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:31.043+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:31.047 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:31.043+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:31.193 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:31.191+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:31.194 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:31.191+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:31.194 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:31.191+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:31.194 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:31.191+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:31.278 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:31.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:31.279 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:31.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:31.279 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:31.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:31.279 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:31.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:31.700 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:31.699+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:31.700 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:31.699+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:31.701 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:31.699+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:31.701 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:31.699+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:32.034 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:32.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:32.034 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:32.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:32.034 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:32.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:32.034 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:32.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:32.161 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:32.163+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:32.161 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:32.163+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:32.161 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:32.163+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:32.161 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:32.163+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:32.307 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:32.307+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:32.308 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:32.307+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:32.308 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:32.307+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:32.308 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:32.307+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:32.746 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:32.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:32.746 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:32.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:32.746 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:32.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:32.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:32.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:33.011 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:33.011+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:33.011 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:33.011+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:33.011 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:33.011+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:33.011 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:33.011+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:33.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:33.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:33.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:33.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:33.132 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:33.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:33.132 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:33.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:33.326 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:33.327+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:33.326 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:33.327+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:33.326 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:33.327+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:33.327 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:33.327+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:33.329 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:57:33.327+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 240 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:57:33.743 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:33.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:33.743 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:33.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:33.743 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:33.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:33.744 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:33.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:33.972 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:33.975+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:33.973 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:33.975+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:33.973 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:33.975+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:33.973 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:33.975+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:34.129 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:34.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:34.129 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:34.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:34.130 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:34.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:34.130 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:34.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:34.312 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:34.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:34.312 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:34.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:34.312 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:34.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:34.313 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:34.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:34.718 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:34.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:34.718 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:34.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:34.718 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:34.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:34.719 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:34.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:34.930 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:34.931+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:34.931 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:34.931+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:34.931 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:34.931+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:34.931 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:34.931+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:35.129 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:35.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:35.130 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:35.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:35.130 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:35.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:35.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:35.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:35.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:35.351+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:35.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:35.351+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:35.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:35.351+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:35.353 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:35.351+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:35.724 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:35.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:35.725 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:35.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:35.725 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:35.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:35.725 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:35.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:35.975 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:35.971+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:35.975 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:35.975+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:35.975 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:35.975+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:35.976 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:35.975+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:36.096 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:36.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:36.096 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:36.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:36.097 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:36.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:36.097 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:36.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:36.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:36.395+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:36.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:36.395+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:36.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:36.395+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:36.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:36.395+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:36.741 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:36.739+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:36.742 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:36.739+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:36.742 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:36.739+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:36.742 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:36.739+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:36.998 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:36.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:36.999 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:36.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:36.999 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:36.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:36.999 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:36.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:37.088 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:37.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:37.088 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:37.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:37.089 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:37.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:37.089 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:37.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:37.390 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:37.387+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:37.391 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:37.387+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:37.391 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:37.387+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:37.391 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:37.387+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:37.708 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:37.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:37.709 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:37.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:37.709 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:37.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:37.709 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:37.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:38.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:38.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:38.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:38.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:38.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:38.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:38.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:38.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:38.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:38.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:38.103 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:38.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:38.103 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:38.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:38.103 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:38.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:38.331 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:57:38.327+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 270 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:57:38.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:38.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:38.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:38.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:38.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:38.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:38.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:38.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:38.749 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:38.751+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:38.750 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:38.751+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:38.750 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:38.751+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:38.750 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:38.751+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:39.029 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:39.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:39.029 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:39.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:39.030 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:39.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:39.030 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:39.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:39.030 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:39.031+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:39.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:39.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:39.129 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:39.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:39.129 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:39.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:39.129 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:39.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:39.335 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:39.331+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:39.335 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:39.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:39.335 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:39.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:39.336 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:39.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:39.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:39.775+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:39.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:39.775+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:39.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:39.775+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:39.775 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:39.775+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:39.983 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:39.983+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:39.983 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:39.983+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:39.984 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:39.983+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:39.984 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:39.983+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:39.984 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:39.983+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:40.160 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:40.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:40.160 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:40.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:40.161 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:40.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:40.161 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:40.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:40.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:40.295+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:40.297 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:40.295+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:40.297 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:40.295+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:40.297 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:40.295+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:40.824 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:40.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:40.824 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:40.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:40.824 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:40.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:40.824 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:40.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:40.951 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:40.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:40.951 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:40.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:40.951 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:40.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:40.952 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:40.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:40.952 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:40.951+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:41.180 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:41.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:41.181 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:41.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:41.181 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:41.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:41.181 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:41.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:41.267 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:41.267+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:41.267 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:41.267+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:41.268 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:41.267+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:41.268 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:41.267+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:41.817 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:41.815+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:41.817 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:41.815+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:41.817 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:41.815+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:41.818 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:41.815+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:41.992 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:41.991+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:41.992 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:41.991+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:41.992 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:41.991+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:41.992 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:41.991+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:41.993 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:41.991+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:42.188 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:42.187+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:42.188 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:42.187+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:42.188 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:42.187+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:42.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:42.187+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:42.287 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:42.287+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:42.288 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:42.287+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:42.288 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:42.287+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:42.288 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:42.287+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:42.824 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:42.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:42.824 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:42.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:42.824 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:42.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:42.824 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:42.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:42.947 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:42.943+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:42.948 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:42.943+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:42.948 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:42.943+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:42.948 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:42.943+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:42.948 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:42.947+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:43.154 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:43.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:43.154 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:43.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:43.155 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:43.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:43.155 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:43.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:43.332 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:57:43.331+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 300 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:57:43.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:43.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:43.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:43.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:43.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:43.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:43.342 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:43.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:43.862 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:43.863+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:43.862 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:43.863+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:43.863 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:43.863+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:43.863 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:43.863+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:43.983 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:43.979+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:43.983 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:43.979+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:43.983 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:43.979+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:43.984 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:43.979+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:43.984 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:43.979+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:44.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:44.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:44.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:44.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:44.180 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:44.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:44.180 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:44.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:44.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:44.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:44.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:44.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:44.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:44.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:44.316 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:44.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:44.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:44.847+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:44.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:44.847+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:44.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:44.847+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:44.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:44.847+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:44.985 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:44.987+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:44.986 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:44.987+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:44.986 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:44.987+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:44.986 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:44.987+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:44.986 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:44.987+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:45.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:45.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:45.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:45.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:45.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:45.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:45.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:45.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:45.284 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:45.283+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:45.285 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:45.283+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:45.285 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:45.283+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:45.285 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:45.283+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:45.854 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:45.855+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:45.854 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:45.855+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:45.855 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:45.855+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:45.855 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:45.855+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:45.982 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:45.979+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:45.982 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:45.979+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:45.982 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:45.979+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:45.983 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:45.979+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:45.983 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:45.979+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:46.146 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:46.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:46.147 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:46.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:46.147 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:46.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:46.147 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:46.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:46.309 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:46.307+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:46.310 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:46.307+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:46.310 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:46.307+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:46.310 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:46.307+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:46.865 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:46.867+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:46.865 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:46.867+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:46.866 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:46.867+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:46.866 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:46.867+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:47.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:47.015+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:47.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:47.015+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:47.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:47.015+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:47.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:47.015+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:47.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:47.015+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:47.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:47.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:47.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:47.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:47.145 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:47.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:47.145 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:47.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:47.357 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:47.359+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:47.357 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:47.359+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:47.357 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:47.359+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:47.357 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:47.359+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:47.891 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:47.891+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:47.891 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:47.891+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:47.892 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:47.891+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:47.892 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:47.891+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:48.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:48.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:48.022 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:48.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:48.022 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:48.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:48.022 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:48.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:48.022 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:48.019+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:48.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:48.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:48.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:48.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:48.132 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:48.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:48.132 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:48.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:48.320 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:48.319+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:48.320 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:48.319+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:48.321 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:48.319+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:48.321 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:48.319+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:48.333 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:57:48.331+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 329 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:57:48.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:48.903+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:48.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:48.903+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:48.905 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:48.903+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:48.905 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:48.903+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:49.066 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:49.067+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:49.066 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:49.067+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:49.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:49.067+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:49.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:49.067+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:49.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:49.067+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:49.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:49.163+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:49.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:49.163+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:49.167 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:49.163+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:49.167 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:49.163+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:49.349 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:49.351+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:49.349 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:49.351+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:49.349 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:49.351+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:49.349 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:49.351+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:49.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:49.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:49.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:49.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:49.926 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:49.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:49.926 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:49.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:50.035 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:50.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:50.035 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:50.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:50.035 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:50.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:50.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:50.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:50.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:50.031+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:50.143 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:50.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:50.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:50.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:50.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:50.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:50.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:50.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:50.394 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:50.391+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:50.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:50.391+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:50.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:50.391+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:50.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:50.391+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:50.957 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:50.959+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:50.958 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:50.959+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:50.958 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:50.959+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:50.958 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:50.959+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:51.038 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:51.039+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:51.038 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:51.039+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:51.038 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:51.039+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:51.038 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:51.039+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:51.039 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:51.039+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:51.160 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:51.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:51.160 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:51.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:51.160 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:51.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:51.161 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:51.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:51.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:51.403+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:51.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:51.403+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:51.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:51.403+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:51.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:51.403+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:51.919 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:51.919+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:51.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:51.919+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:51.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:51.919+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:51.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:51.919+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:52.037 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:52.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:52.037 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:52.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:52.038 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:52.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:52.038 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:52.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:52.038 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:52.035+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:52.127 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:52.123+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:52.127 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:52.123+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:52.127 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:52.123+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:52.127 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:52.123+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:52.415 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:52.415+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:52.416 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:52.415+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:52.416 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:52.415+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:52.416 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:52.415+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:52.935 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:52.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:52.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:52.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:52.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:52.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:52.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:52.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:53.087 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:53.083+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:53.087 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:53.083+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:53.087 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:53.083+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:53.087 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:53.083+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:53.088 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:53.083+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:53.139 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:53.135+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:53.139 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:53.139+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:53.139 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:53.139+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:53.140 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:53.139+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:53.334 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:57:53.335+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 359 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:57:53.442 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:53.439+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:53.442 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:53.439+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:53.442 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:53.439+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:53.443 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:53.439+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:53.926 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:53.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:53.926 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:53.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:53.926 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:53.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:53.926 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:53.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:54.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:54.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:54.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:54.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:54.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:54.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:54.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:54.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:54.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:54.103+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:54.117 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:54.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:54.117 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:54.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:54.117 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:54.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:54.117 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:54.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:54.485 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:54.487+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:54.485 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:54.487+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:54.485 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:54.487+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:54.486 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:54.487+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:54.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:54.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:54.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:54.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:54.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:54.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:54.937 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:54.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:55.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:55.079+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:55.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:55.079+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:55.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:55.079+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:55.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:55.079+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:55.079 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:55.079+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:55.101 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:55.099+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:55.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:55.099+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:55.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:55.099+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:55.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:55.099+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:55.447 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:55.443+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:55.447 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:55.443+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:55.447 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:55.443+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:55.448 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:55.447+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:55.929 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:55.927+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:55.929 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:55.927+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:55.930 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:55.927+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:55.930 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:55.927+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:56.091 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:56.087+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:56.091 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:56.087+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:56.091 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:56.087+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:56.092 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:56.087+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:56.092 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:56.091+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:56.111 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:56.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:56.111 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:56.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:56.111 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:56.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:56.111 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:56.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:56.475 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:56.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:56.475 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:56.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:56.475 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:56.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:56.475 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:56.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:56.911 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:56.911+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:56.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:56.911+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:56.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:56.911+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:56.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:56.911+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:57.071 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:57.071+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:57.072 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:57.071+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:57.072 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:57.071+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:57.072 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:57.071+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:57.072 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:57.071+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:57.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:57.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:57.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:57.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:57.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:57.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:57.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:57.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:57.449 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:57.447+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:57.450 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:57.447+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:57.450 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:57.447+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:57.450 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:57.447+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:57.935 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:57.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:57.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:57.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:57.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:57.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:57.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:57.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:58.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:58.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:58.061 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:58.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:58.061 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:58.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:58.061 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:58.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:58.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:58.119+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:58.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:58.119+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:58.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:58.119+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:58.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:58.119+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:58.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:58.119+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:58.334 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:57:58.335+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 389 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:57:58.476 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:58.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:58.477 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:58.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:58.477 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:58.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:58.477 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:58.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:58.896 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:58.895+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:58.896 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:58.895+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:58.896 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:58.895+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:58.896 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:58.895+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:59.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:59.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:59.103 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:59.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:59.103 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:59.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:59.103 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:57:59.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:57:59.132 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:59.131+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:59.132 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:59.131+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:59.132 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:59.131+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:59.132 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:59.131+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:57:59.133 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:57:59.131+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:57:59.520 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:59.523+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:59.521 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:59.523+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:59.521 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:59.523+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:59.521 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:57:59.523+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:57:59.931 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:59.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:59.931 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:59.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:59.932 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:59.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:57:59.932 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:57:59.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:00.137 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:00.135+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:00.138 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:00.135+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:00.138 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:00.135+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:00.138 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:00.135+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:00.142 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:00.139+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:00.143 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:00.139+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:00.143 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:00.139+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:00.143 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:00.139+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:00.143 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:00.139+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:00.512 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:00.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:00.512 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:00.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:00.513 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:00.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:00.513 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:00.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:00.928 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:00.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:00.928 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:00.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:00.928 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:00.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:00.928 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:00.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:01.132 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:01.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:01.132 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:01.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:01.132 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:01.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:01.132 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:01.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:01.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:01.151+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:01.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:01.151+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:01.152 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:01.151+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:01.152 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:01.151+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:01.152 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:01.151+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:01.492 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:01.495+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:01.493 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:01.495+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:01.493 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:01.495+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:01.493 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:01.495+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:01.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:01.883+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:01.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:01.883+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:01.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:01.883+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:01.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:01.883+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:02.104 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:02.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:02.105 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:02.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:02.105 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:02.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:02.105 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:02.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:02.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:02.147+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:02.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:02.147+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:02.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:02.147+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:02.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:02.147+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:02.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:02.147+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:02.529 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:02.527+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:02.530 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:02.527+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:02.530 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:02.527+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:02.530 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:02.527+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:02.921 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:02.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:02.922 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:02.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:02.922 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:02.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:02.922 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:02.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:03.078 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:03.078+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:03.078 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:03.078+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:03.078 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:03.078+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:03.079 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:03.078+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:03.185 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:03.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:03.186 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:03.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:03.186 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:03.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:03.186 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:03.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:03.186 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:03.186+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:03.335 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:58:03.334+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 419 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:58:03.532 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:03.534+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:03.533 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:03.534+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:03.533 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:03.534+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:03.533 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:03.534+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:03.957 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:03.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:03.957 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:03.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:03.957 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:03.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:03.958 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:03.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:04.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:04.122+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:04.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:04.122+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:04.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:04.122+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:04.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:04.122+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:04.175 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:04.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:04.176 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:04.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:04.176 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:04.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:04.176 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:04.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:04.176 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:04.174+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:04.497 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:04.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:04.497 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:04.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:04.498 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:04.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:04.498 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:04.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:04.911 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:04.910+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:04.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:04.910+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:04.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:04.910+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:04.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:04.910+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:05.104 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:05.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:05.105 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:05.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:05.105 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:05.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:05.105 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:05.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:05.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:05.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:05.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:05.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:05.175 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:05.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:05.175 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:05.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:05.175 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:05.170+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:05.518 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:05.514+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:05.518 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:05.514+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:05.519 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:05.514+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:05.519 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:05.514+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:05.916 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:05.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:05.916 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:05.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:05.917 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:05.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:05.917 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:05.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:06.147 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:06.146+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:06.147 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:06.146+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:06.148 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:06.146+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:06.148 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:06.146+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:06.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:06.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:06.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:06.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:06.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:06.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:06.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:06.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:06.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:06.210+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:06.497 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:06.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:06.498 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:06.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:06.498 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:06.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:06.498 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:06.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:06.878 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:06.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:06.879 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:06.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:06.879 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:06.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:06.879 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:06.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:07.101 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:07.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:07.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:07.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:07.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:07.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:07.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:07.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:07.208 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:07.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:07.209 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:07.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:07.209 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:07.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:07.209 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:07.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:07.209 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:07.210+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:07.507 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:07.506+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:07.507 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:07.506+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:07.507 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:07.506+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:07.508 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:07.506+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:07.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:07.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:07.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:07.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:07.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:07.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:07.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:07.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:08.052 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:08.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:08.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:08.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:08.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:08.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:08.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:08.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:08.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:08.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:08.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:08.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:08.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:08.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:08.223 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:08.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:08.223 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:08.218+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:08.336 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:58:08.334+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 448 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:58:08.499 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:08.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:08.499 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:08.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:08.500 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:08.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:08.500 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:08.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:08.911 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:08.906+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:08.911 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:08.910+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:08.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:08.910+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:08.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:08.910+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:09.065 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:09.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:09.065 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:09.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:09.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:09.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:09.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:09.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:09.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:09.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:09.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:09.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:09.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:09.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:09.217 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:09.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:09.217 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:09.218+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:09.471 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:09.470+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:09.471 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:09.470+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:09.471 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:09.470+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:09.472 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:09.470+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:09.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:09.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:09.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:09.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:09.883 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:09.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:09.883 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:09.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:10.038 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:10.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:10.038 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:10.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:10.038 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:10.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:10.039 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:10.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:10.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:10.190+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:10.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:10.190+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:10.194 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:10.190+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:10.194 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:10.190+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:10.194 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:10.190+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:10.452 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:10.450+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:10.453 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:10.450+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:10.453 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:10.450+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:10.453 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:10.450+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:10.926 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:10.926+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:10.926 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:10.926+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:10.926 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:10.926+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:10.926 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:10.926+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:11.031 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:11.026+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:11.032 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:11.026+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:11.032 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:11.026+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:11.032 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:11.026+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:11.188 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:11.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:11.188 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:11.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:11.188 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:11.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:11.188 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:11.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:11.189 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:11.186+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:11.438 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:11.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:11.438 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:11.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:11.438 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:11.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:11.439 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:11.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:11.878 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:11.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:11.879 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:11.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:11.879 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:11.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:11.879 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:11.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:12.044 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:12.042+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:12.045 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:12.042+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:12.045 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:12.042+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:12.045 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:12.042+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:12.188 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:12.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:12.189 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:12.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:12.189 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:12.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:12.189 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:12.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:12.189 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:12.186+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:12.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:12.406+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:12.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:12.406+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:12.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:12.406+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:12.406 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:12.406+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:12.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:12.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:12.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:12.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:12.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:12.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:12.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:12.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:13.022 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:13.022+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:13.022 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:13.022+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:13.023 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:13.022+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:13.023 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:13.022+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:13.211 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:13.206+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:13.211 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:13.206+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:13.211 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:13.206+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:13.211 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:13.206+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:13.212 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:13.210+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:13.337 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:58:13.334+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 478 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:58:13.385 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:13.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:13.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:13.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:13.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:13.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:13.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:13.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:13.893 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:13.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:13.893 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:13.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:13.893 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:13.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:13.894 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:13.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:14.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:14.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:14.054 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:14.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:14.054 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:14.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:14.054 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:14.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:14.164 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:14.166+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:14.164 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:14.166+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:14.165 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:14.166+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:14.165 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:14.166+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:14.165 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:14.166+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:14.391 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:14.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:14.392 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:14.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:14.392 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:14.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:14.392 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:14.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:14.937 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:14.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:14.938 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:14.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:14.938 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:14.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:14.938 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:14.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:15.013 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:15.014+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:15.013 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:15.014+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:15.014 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:15.014+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:15.014 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:15.014+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:15.205 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:15.202+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:15.206 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:15.202+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:15.206 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:15.202+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:15.206 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:15.202+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:15.206 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:15.202+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:15.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:15.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:15.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:15.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:15.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:15.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:15.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:15.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:15.931 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:15.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:15.932 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:15.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:15.932 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:15.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:15.932 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:15.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:16.014 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:16.014+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:16.014 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:16.014+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:16.014 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:16.014+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:16.015 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:16.014+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:16.161 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:16.158+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:16.162 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:16.158+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:16.162 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:16.158+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:16.162 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:16.158+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:16.163 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:16.158+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:16.446 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:16.442+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:16.447 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:16.442+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:16.447 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:16.442+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:16.447 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:16.442+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:16.935 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:16.934+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:16.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:16.934+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:16.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:16.934+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:16.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:16.934+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:17.058 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:17.058+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:17.059 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:17.058+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:17.059 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:17.058+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:17.059 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:17.058+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:17.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:17.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:17.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:17.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:17.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:17.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:17.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:17.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:17.152 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:17.146+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:17.488 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:17.490+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:17.489 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:17.490+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:17.489 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:17.490+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:17.489 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:17.490+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:17.928 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:17.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:17.929 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:17.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:17.929 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:17.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:17.929 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:17.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:18.070 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:18.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:18.070 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:18.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:18.071 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:18.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:18.071 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:18.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:18.200 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:18.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:18.200 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:18.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:18.200 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:18.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:18.201 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:18.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:18.201 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:18.198+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:18.338 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:58:18.334+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 508 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:58:18.502 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:18.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:18.502 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:18.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:18.502 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:18.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:18.503 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:18.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:18.955 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:18.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:18.955 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:18.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:18.955 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:18.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:18.956 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:18.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:19.081 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:19.078+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:19.081 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:19.078+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:19.081 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:19.078+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:19.082 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:19.078+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:19.236 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:19.238+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:19.236 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:19.238+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:19.236 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:19.238+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:19.236 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:19.238+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:19.237 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:19.238+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:19.523 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:19.522+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:19.523 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:19.522+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:19.524 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:19.522+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:19.524 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:19.522+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:19.952 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:19.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:19.953 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:19.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:19.953 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:19.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:19.953 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:19.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:20.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:20.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:20.067 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:20.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:20.067 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:20.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:20.067 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:20.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:20.256 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:20.254+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:20.256 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:20.254+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:20.256 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:20.254+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:20.256 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:20.254+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:20.257 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:20.254+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:20.528 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:20.530+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:20.528 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:20.530+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:20.528 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:20.530+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:20.528 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:20.530+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:20.957 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:20.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:20.957 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:20.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:20.957 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:20.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:20.958 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:20.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:21.099 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:21.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:21.100 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:21.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:21.100 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:21.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:21.100 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:21.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:21.220 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:21.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:21.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:21.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:21.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:21.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:21.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:21.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:21.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:21.218+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:21.518 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:21.518+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:21.519 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:21.518+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:21.519 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:21.518+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:21.519 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:21.518+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:21.963 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:21.962+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:21.964 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:21.962+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:21.964 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:21.962+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:21.964 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:21.962+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:22.106 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:22.106+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:22.106 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:22.106+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:22.107 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:22.106+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:22.107 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:22.106+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:22.208 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:22.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:22.208 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:22.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:22.209 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:22.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:22.209 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:22.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:22.209 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:22.210+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:22.497 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:22.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:22.498 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:22.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:22.498 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:22.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:22.498 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:22.498+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:22.990 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:22.990+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:22.990 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:22.990+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:22.990 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:22.990+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:22.990 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:22.990+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:23.143 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:23.146+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:23.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:23.146+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:23.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:23.146+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:23.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:23.146+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:23.252 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:23.254+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:23.252 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:23.254+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:23.252 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:23.254+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:23.252 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:23.254+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:23.253 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:23.254+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:23.338 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:58:23.338+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 538 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:58:23.528 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:23.526+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:23.528 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:23.526+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:23.529 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:23.526+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:23.529 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:23.526+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:23.972 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:23.970+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:23.973 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:23.970+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:23.973 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:23.970+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:23.973 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:23.970+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:24.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:24.178+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:24.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:24.178+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:24.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:24.178+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:24.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:24.178+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:24.246 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:24.246+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:24.247 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:24.246+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:24.247 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:24.246+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:24.247 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:24.246+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:24.247 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:24.246+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:24.502 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:24.502+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:24.502 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:24.502+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:24.503 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:24.502+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:24.503 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:24.502+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:24.957 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:24.958+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:24.957 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:24.958+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:24.958 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:24.958+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:24.958 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:24.958+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:25.218 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:25.218+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:25.218 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:25.218+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:25.219 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:25.218+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:25.219 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:25.218+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:25.246 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:25.246+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:25.247 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:25.246+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:25.247 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:25.246+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:25.247 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:25.246+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:25.247 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:25.246+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:25.516 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:25.518+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:25.516 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:25.518+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:25.517 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:25.518+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:25.517 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:25.518+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:25.937 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:25.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:25.937 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:25.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:25.938 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:25.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:25.938 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:25.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:26.183 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:26.182+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:26.183 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:26.182+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:26.183 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:26.182+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:26.184 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:26.182+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:26.280 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:26.282+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:26.281 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:26.282+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:26.281 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:26.282+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:26.281 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:26.282+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:26.281 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:26.282+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:26.554 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:26.550+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:26.555 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:26.550+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:26.555 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:26.550+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:26.555 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:26.550+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:26.914 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:26.910+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:26.914 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:26.910+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:26.914 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:26.910+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:26.915 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:26.910+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:27.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:27.154+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:27.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:27.154+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:27.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:27.154+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:27.157 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:27.154+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:27.289 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:27.290+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:27.290 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:27.290+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:27.290 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:27.290+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:27.290 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:27.290+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:27.290 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:27.290+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:27.581 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:27.582+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:27.582 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:27.582+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:27.582 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:27.582+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:27.582 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:27.582+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:27.956 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:27.958+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:27.956 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:27.958+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:27.957 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:27.958+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:27.957 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:27.958+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:28.149 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:28.150+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:28.150 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:28.150+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:28.150 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:28.150+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:28.150 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:28.150+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:28.265 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:28.266+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:28.265 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:28.266+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:28.265 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:28.266+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:28.266 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:28.266+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:28.266 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:28.266+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:28.339 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:58:28.338+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 567 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:58:28.603 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:28.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:28.603 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:28.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:28.603 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:28.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:28.604 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:28.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:28.941 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:28.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:28.941 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:28.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:28.941 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:28.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:28.942 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:28.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:29.116 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:29.118+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:29.116 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:29.118+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:29.116 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:29.118+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:29.117 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:29.118+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:29.292 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:29.290+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:29.292 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:29.290+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:29.292 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:29.290+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:29.292 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:29.290+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:29.293 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:29.290+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:29.602 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:29.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:29.603 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:29.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:29.603 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:29.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:29.603 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:29.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:29.918 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:29.918+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:29.918 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:29.918+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:29.919 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:29.918+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:29.919 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:29.918+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:30.070 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:30.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:30.070 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:30.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:30.071 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:30.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:30.071 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:30.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:30.257 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:30.258+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:30.258 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:30.258+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:30.258 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:30.258+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:30.258 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:30.258+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:30.258 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:30.258+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:30.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:30.614+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:30.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:30.614+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:30.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:30.614+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:30.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:30.614+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:30.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:30.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:30.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:30.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:30.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:30.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:30.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:30.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:31.097 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:31.094+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:31.097 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:31.094+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:31.097 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:31.094+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:31.098 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:31.094+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:31.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:31.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:31.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:31.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:31.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:31.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:31.236 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:31.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:31.236 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:31.234+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:31.650 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:31.650+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:31.651 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:31.650+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:31.651 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:31.650+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:31.651 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:31.650+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:31.897 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:31.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:31.897 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:31.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:31.898 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:31.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:31.898 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:31.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:32.048 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:32.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:32.049 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:32.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:32.049 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:32.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:32.049 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:32.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:32.220 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:32.222+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:32.220 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:32.222+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:32.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:32.222+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:32.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:32.222+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:32.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:32.222+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:32.641 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:32.642+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:32.642 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:32.642+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:32.642 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:32.642+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:32.642 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:32.642+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:32.850 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:32.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:32.850 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:32.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:32.851 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:32.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:32.851 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:32.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:33.033 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:33.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:33.034 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:33.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:33.034 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:33.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:33.034 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:33.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:33.179 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:33.182+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:33.180 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:33.182+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:33.180 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:33.182+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:33.181 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:33.182+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:33.181 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:33.182+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:33.341 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:58:33.338+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 597 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:58:33.624 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:33.626+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:33.625 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:33.626+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:33.625 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:33.626+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:33.626 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:33.626+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:33.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:33.890+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:33.890 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:33.890+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:33.890 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:33.890+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:33.890 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:33.890+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:34.027 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:34.030+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:34.028 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:34.030+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:34.029 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:34.030+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:34.029 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:34.030+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:34.189 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:34.190+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:34.190 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:34.190+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:34.190 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:34.190+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:34.190 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:34.190+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:34.190 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:34.190+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:34.609 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:34.610+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:34.610 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:34.610+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:34.610 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:34.610+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:34.610 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:34.610+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:34.870 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:34.870+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:34.871 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:34.870+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:34.871 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:34.870+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:34.871 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:34.870+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:35.015 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:35.014+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:35.015 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:35.014+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:35.015 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:35.014+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:35.015 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:35.014+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:35.154 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:35.154+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:35.154 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:35.154+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:35.154 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:35.154+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:35.154 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:35.154+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:35.155 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:35.154+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:35.578 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:35.578+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:35.579 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:35.578+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:35.579 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:35.578+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:35.579 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:35.578+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:35.898 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:35.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:35.899 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:35.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:35.899 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:35.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:35.899 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:35.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:36.059 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:36.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:36.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:36.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:36.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:36.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:36.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:36.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:36.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:36.194+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:36.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:36.194+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:36.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:36.194+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:36.194 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:36.194+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:36.194 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:36.194+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:36.536 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:36.538+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:36.537 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:36.538+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:36.537 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:36.538+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:36.537 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:36.538+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:36.866 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:36.866+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:36.866 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:36.866+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:36.867 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:36.866+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:36.867 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:36.866+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:37.048 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:37.050+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:37.049 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:37.050+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:37.049 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:37.050+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:37.049 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:37.050+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:37.209 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:37.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:37.209 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:37.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:37.210 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:37.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:37.210 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:37.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:37.210 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:37.210+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:37.491 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:37.494+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:37.492 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:37.494+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:37.492 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:37.494+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:37.492 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:37.494+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:37.845 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:37.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:37.845 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:37.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:37.846 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:37.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:37.846 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:37.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:38.005 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:38.006+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:38.006 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:38.006+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:38.006 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:38.006+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:38.007 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:38.006+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:38.192 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:38.194+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:38.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:38.194+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:38.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:38.194+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:38.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:38.194+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:38.194 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:38.194+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:38.342 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:58:38.342+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 627 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:58:38.500 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:38.502+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:38.500 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:38.502+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:38.500 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:38.502+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:38.501 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:38.502+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:38.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:38.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:38.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:38.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:38.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:38.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:38.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:38.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:39.010 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:39.010+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:39.010 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:39.010+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:39.011 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:39.010+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:39.011 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:39.010+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:39.166 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:39.166+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:39.167 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:39.166+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:39.167 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:39.166+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:39.167 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:39.166+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:39.167 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:39.166+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:39.517 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:39.518+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:39.517 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:39.518+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:39.518 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:39.518+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:39.518 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:39.518+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:39.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:39.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:39.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:39.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:39.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:39.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:39.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:39.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:39.999 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:40.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:39.999 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:40.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:40.000 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:40.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:40.000 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:40.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:40.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:40.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:40.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:40.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:40.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:40.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:40.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:40.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:40.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:40.122+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:40.539 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:40.538+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:40.539 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:40.538+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:40.539 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:40.538+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:40.540 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:40.538+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:40.950 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:40.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:40.951 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:40.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:40.951 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:40.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:40.951 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:40.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:40.960 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:40.962+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:40.960 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:40.962+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:40.960 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:40.962+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:40.961 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:40.962+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:41.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:41.150+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:41.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:41.150+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:41.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:41.150+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:41.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:41.150+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:41.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:41.150+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:41.578 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:41.578+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:41.578 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:41.578+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:41.579 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:41.578+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:41.579 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:41.578+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:41.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:41.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:41.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:41.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:41.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:41.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:41.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:41.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:41.993 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:41.994+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:41.993 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:41.994+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:41.993 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:41.994+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:41.993 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:41.994+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:42.156 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:42.158+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:42.157 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:42.158+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:42.157 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:42.158+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:42.157 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:42.158+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:42.157 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:42.158+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:42.560 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:42.562+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:42.560 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:42.562+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:42.561 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:42.562+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:42.561 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:42.562+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:42.927 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:42.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:42.927 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:42.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:42.927 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:42.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:42.928 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:42.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:42.947 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:42.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:42.948 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:42.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:42.948 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:42.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:42.948 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:42.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:43.186 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:43.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:43.187 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:43.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:43.187 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:43.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:43.187 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:43.186+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:43.187 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:43.186+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:43.343 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:58:43.346+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 656 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:58:43.603 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:43.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:43.603 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:43.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:43.604 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:43.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:43.604 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:43.606+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:43.899 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:43.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:43.900 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:43.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:43.900 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:43.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:43.900 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:43.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:43.903 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:43.906+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:43.903 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:43.906+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:43.903 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:43.906+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:43.903 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:43.906+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:44.176 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:44.178+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:44.176 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:44.178+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:44.177 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:44.178+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:44.177 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:44.178+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:44.177 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:44.178+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:44.640 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:44.642+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:44.640 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:44.642+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:44.640 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:44.642+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:44.641 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:44.642+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:44.928 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:44.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:44.928 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:44.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:44.928 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:44.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:44.929 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:44.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:44.936 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:44.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:44.936 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:44.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:44.936 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:44.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:44.936 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:44.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:45.171 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:45.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:45.171 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:45.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:45.171 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:45.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:45.171 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:45.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:45.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:45.170+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:45.602 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:45.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:45.603 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:45.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:45.603 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:45.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:45.603 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:45.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:45.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:45.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:45.937 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:45.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:45.937 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:45.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:45.937 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:45.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:45.950 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:45.950+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:45.950 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:45.950+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:45.950 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:45.950+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:45.950 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:45.950+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:46.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:46.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:46.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:46.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:46.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:46.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:46.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:46.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:46.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:46.218+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:46.590 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:46.590+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:46.590 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:46.590+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:46.590 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:46.590+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:46.590 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:46.590+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:46.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:46.890+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:46.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:46.890+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:46.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:46.890+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:46.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:46.890+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:46.922 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:46.922+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:46.923 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:46.922+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:46.923 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:46.922+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:46.923 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:46.922+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:47.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:47.214+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:47.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:47.214+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:47.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:47.214+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:47.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:47.214+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:47.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:47.214+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:47.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:47.618+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:47.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:47.618+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:47.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:47.618+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:47.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:47.618+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:47.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:47.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:47.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:47.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:47.914 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:47.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:47.914 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:47.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:47.950 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:47.950+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:47.950 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:47.950+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:47.951 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:47.950+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:47.951 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:47.950+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:48.203 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:48.202+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:48.203 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:48.206+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:48.203 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:48.206+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:48.204 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:48.206+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:48.204 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:48.206+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:48.344 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:58:48.346+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 686 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:58:48.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:48.618+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:48.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:48.618+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:48.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:48.618+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:48.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:48.618+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:48.892 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:48.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:48.892 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:48.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:48.892 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:48.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:48.893 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:48.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:48.915 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:48.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:48.915 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:48.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:48.916 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:48.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:48.916 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:48.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:49.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:49.194+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:49.194 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:49.194+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:49.194 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:49.194+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:49.194 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:49.194+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:49.194 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:49.194+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:49.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:49.646+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:49.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:49.646+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:49.645 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:49.646+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:49.645 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:49.646+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:49.854 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:49.854+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:49.855 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:49.854+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:49.855 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:49.854+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:49.855 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:49.854+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:49.937 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:49.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:49.937 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:49.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:49.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:49.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:49.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:49.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:50.171 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:50.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:50.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:50.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:50.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:50.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:50.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:50.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:50.173 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:50.174+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:50.679 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:50.682+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:50.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:50.682+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:50.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:50.682+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:50.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:50.682+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:50.871 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:50.874+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:50.872 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:50.874+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:50.872 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:50.874+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:50.872 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:50.874+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:50.928 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:50.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:50.929 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:50.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:50.929 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:50.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:50.929 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:50.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:51.195 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:51.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:51.195 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:51.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:51.195 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:51.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:51.196 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:51.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:51.196 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:51.198+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:51.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:51.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:51.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:51.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:51.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:51.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:51.663 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:51.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:51.885 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:51.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:51.885 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:51.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:51.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:51.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:51.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:51.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:51.958 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:51.958+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:51.958 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:51.958+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:51.959 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:51.958+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:51.959 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:51.958+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:52.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:52.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:52.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:52.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:52.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:52.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:52.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:52.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:52.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:52.234+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:52.668 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:52.670+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:52.668 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:52.670+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:52.669 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:52.670+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:52.669 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:52.670+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:52.878 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:52.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:52.878 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:52.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:52.879 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:52.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:52.879 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:52.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:52.926 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:52.926+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:52.926 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:52.926+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:52.926 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:52.926+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:52.927 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:52.926+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:53.219 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:53.222+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:53.220 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:53.222+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:53.220 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:53.222+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:53.220 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:53.222+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:53.220 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:53.222+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:53.345 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:58:53.346+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 715 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:58:53.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:53.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:53.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:53.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:53.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:53.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:53.663 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:53.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:53.842 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:53.842+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:53.842 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:53.842+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:53.843 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:53.842+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:53.843 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:53.842+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:53.918 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:53.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:53.919 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:53.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:53.919 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:53.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:53.919 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:53.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:54.197 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:54.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:54.198 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:54.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:54.198 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:54.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:54.198 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:54.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:54.198 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:54.198+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:54.612 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:54.614+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:54.612 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:54.614+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:54.612 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:54.614+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:54.613 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:54.614+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:54.797 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:54.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:54.797 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:54.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:54.797 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:54.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:54.797 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:54.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:54.890 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:54.890+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:54.890 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:54.890+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:54.890 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:54.890+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:54.890 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:54.890+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:55.233 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:55.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:55.233 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:55.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:55.233 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:55.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:55.233 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:55.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:55.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:55.234+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:55.631 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:55.634+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:55.631 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:55.634+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:55.631 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:55.634+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:55.632 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:55.634+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:55.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:55.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:55.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:55.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:55.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:55.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:55.839 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:55.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:55.871 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:55.874+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:55.871 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:55.874+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:55.871 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:55.874+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:55.871 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:55.874+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:56.268 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:56.270+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:56.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:56.270+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:56.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:56.270+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:56.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:56.270+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:56.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:56.270+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:56.620 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:56.622+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:56.621 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:56.622+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:56.621 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:56.622+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:56.621 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:56.622+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:56.837 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:56.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:56.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:56.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:56.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:56.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:56.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:56.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:56.847 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:56.850+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:56.847 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:56.850+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:56.847 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:56.850+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:56.847 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:56.850+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:57.242 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:57.242+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:57.242 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:57.242+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:57.242 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:57.242+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:57.242 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:57.242+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:57.243 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:57.242+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:57.632 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:57.634+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:57.633 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:57.634+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:57.633 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:57.634+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:57.633 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:57.634+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:57.851 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:57.853+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:57.852 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:57.853+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:57.852 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:57.853+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:57.852 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:57.853+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:57.894 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:57.897+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:57.895 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:57.897+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:57.895 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:57.897+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:57.895 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:57.897+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:58.238 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:58.237+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:58.239 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:58.237+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:58.239 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:58.237+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:58.239 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:58.237+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:58.239 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:58.237+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:58.346 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:58:58.345+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 745 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:58:58.623 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:58.625+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:58.623 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:58.625+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:58.624 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:58.625+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:58.624 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:58.625+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:58.815 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:58.813+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:58.816 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:58.813+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:58.816 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:58.813+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:58.816 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:58.813+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:58.919 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:58.921+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:58.920 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:58.921+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:58.920 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:58.921+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:58.920 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:58.921+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:59.248 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:59.249+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:59.249 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:59.249+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:59.249 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:59.249+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:59.249 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:59.249+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:58:59.249 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:58:59.249+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:58:59.605 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:59.605+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:59.606 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:59.605+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:59.606 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:59.605+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:59.606 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:58:59.605+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:58:59.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:59.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:59.849 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:59.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:59.849 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:59.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:59.849 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:58:59.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:58:59.930 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:59.929+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:59.931 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:59.929+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:59.931 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:59.929+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:58:59.931 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:58:59.929+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:00.238 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:00.241+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:00.239 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:00.241+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:00.239 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:00.241+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:00.239 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:00.241+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:00.239 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:00.241+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:00.636 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:00.637+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:00.637 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:00.637+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:00.637 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:00.637+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:00.637 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:00.637+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:00.851 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:00.853+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:00.851 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:00.853+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:00.851 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:00.853+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:00.851 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:00.853+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:00.918 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:00.917+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:00.919 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:00.917+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:00.919 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:00.921+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:00.919 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:00.921+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:01.255 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:01.257+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:01.256 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:01.257+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:01.256 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:01.257+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:01.256 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:01.257+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:01.257 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:01.257+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:01.592 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:01.593+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:01.592 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:01.593+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:01.592 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:01.593+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:01.592 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:01.593+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:01.820 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:01.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:01.820 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:01.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:01.821 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:01.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:01.821 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:01.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:01.928 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:01.929+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:01.928 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:01.929+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:01.928 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:01.929+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:01.928 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:01.929+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:02.275 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:02.277+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:02.276 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:02.277+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:02.276 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:02.277+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:02.276 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:02.277+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:02.276 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:02.277+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:02.564 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:02.565+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:02.564 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:02.565+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:02.565 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:02.565+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:02.565 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:02.565+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:02.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:02.773+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:02.775 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:02.773+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:02.775 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:02.773+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:02.775 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:02.773+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:02.952 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:02.953+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:02.953 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:02.953+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:02.953 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:02.953+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:02.953 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:02.953+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:03.273 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:03.273+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:03.273 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:03.273+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:03.274 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:03.273+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:03.274 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:03.273+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:03.274 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:03.273+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:03.347 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:59:03.345+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 775 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:59:03.592 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:03.593+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:03.592 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:03.593+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:03.593 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:03.593+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:03.593 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:03.593+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:03.728 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:03.729+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:03.728 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:03.729+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:03.729 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:03.729+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:03.729 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:03.729+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:03.987 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:03.989+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:03.988 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:03.989+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:03.988 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:03.989+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:03.988 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:03.989+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:04.260 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:04.253+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:04.260 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:04.253+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:04.260 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:04.253+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:04.261 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:04.253+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:04.261 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:04.253+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:04.586 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:04.585+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:04.586 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:04.585+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:04.587 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:04.585+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:04.587 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:04.585+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:04.696 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:04.697+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:04.697 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:04.697+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:04.697 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:04.697+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:04.697 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:04.697+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:05.008 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:05.009+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:05.009 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:05.009+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:05.009 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:05.009+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:05.009 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:05.009+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:05.288 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:05.289+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:05.289 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:05.289+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:05.289 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:05.289+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:05.289 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:05.289+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:05.289 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:05.289+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:05.632 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:05.633+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:05.633 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:05.633+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:05.633 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:05.633+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:05.633 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:05.633+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:05.714 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:05.713+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:05.715 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:05.713+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:05.715 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:05.713+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:05.715 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:05.713+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:06.045 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:06.045+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:06.045 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:06.045+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:06.046 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:06.045+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:06.046 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:06.045+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:06.243 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:06.245+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:06.243 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:06.245+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:06.243 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:06.245+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:06.243 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:06.245+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:06.244 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:06.245+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:06.639 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:06.641+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:06.640 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:06.641+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:06.640 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:06.641+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:06.640 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:06.641+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:06.667 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:06.669+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:06.667 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:06.669+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:06.667 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:06.669+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:06.668 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:06.669+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:07.093 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:07.093+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:07.093 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:07.093+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:07.094 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:07.093+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:07.094 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:07.093+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:07.211 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:07.209+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:07.211 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:07.209+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:07.211 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:07.209+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:07.211 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:07.209+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:07.212 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:07.213+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:07.669 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:07.669+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:07.670 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:07.669+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:07.670 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:07.669+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:07.670 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:07.669+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:07.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:07.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:07.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:07.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:07.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:07.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:07.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:07.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:08.110 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:08.109+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:08.111 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:08.109+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:08.111 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:08.109+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:08.111 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:08.109+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:08.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:08.173+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:08.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:08.173+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:08.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:08.173+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:08.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:08.173+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:08.173 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:08.173+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:08.348 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:59:08.349+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 804 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:59:08.685 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:08.685+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:08.685 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:08.685+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:08.685 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:08.685+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:08.686 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:08.685+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:08.695 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:08.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:08.695 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:08.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:08.695 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:08.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:08.696 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:08.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:09.080 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:09.081+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:09.081 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:09.081+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:09.081 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:09.081+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:09.081 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:09.081+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:09.219 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:09.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:09.219 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:09.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:09.219 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:09.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:09.219 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:09.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:09.220 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:09.221+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:09.639 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:09.641+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:09.639 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:09.641+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:09.640 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:09.641+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:09.640 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:09.641+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:09.706 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:09.709+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:09.707 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:09.709+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:09.707 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:09.709+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:09.707 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:09.709+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:10.121 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:10.121+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:10.121 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:10.121+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:10.122 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:10.121+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:10.122 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:10.121+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:10.177 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:10.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:10.177 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:10.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:10.177 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:10.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:10.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:10.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:10.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:10.177+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:10.595 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:10.597+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:10.596 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:10.597+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:10.596 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:10.597+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:10.596 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:10.597+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:10.729 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:10.729+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:10.730 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:10.729+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:10.730 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:10.729+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:10.730 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:10.729+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:11.114 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:11.113+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:11.114 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:11.113+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:11.115 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:11.113+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:11.115 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:11.113+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:11.182 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:11.181+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:11.182 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:11.181+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:11.182 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:11.181+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:11.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:11.181+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:11.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:11.181+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:11.553 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:11.553+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:11.553 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:11.553+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:11.553 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:11.553+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:11.554 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:11.553+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:11.717 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:11.717+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:11.718 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:11.717+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:11.718 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:11.717+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:11.718 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:11.717+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:12.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:12.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:12.129 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:12.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:12.129 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:12.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:12.129 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:12.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:12.231 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:12.233+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:12.232 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:12.233+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:12.232 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:12.233+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:12.232 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:12.233+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:12.233 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:12.233+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:12.528 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:12.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:12.529 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:12.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:12.529 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:12.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:12.529 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:12.529+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:12.682 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:12.681+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:12.683 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:12.681+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:12.683 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:12.681+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:12.683 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:12.681+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:13.132 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:13.133+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:13.133 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:13.133+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:13.133 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:13.133+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:13.133 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:13.133+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:13.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:13.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:13.223 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:13.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:13.223 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:13.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:13.223 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:13.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:13.223 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:13.221+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:13.349 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:59:13.349+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 834 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:59:13.505 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:13.505+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:13.505 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:13.505+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:13.505 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:13.505+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:13.505 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:13.505+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:13.720 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:13.721+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:13.720 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:13.721+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:13.720 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:13.721+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:13.721 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:13.721+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:14.152 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:14.153+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:14.153 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:14.153+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:14.153 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:14.153+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:14.153 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:14.153+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:14.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:14.185+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:14.184 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:14.185+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:14.184 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:14.185+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:14.184 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:14.185+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:14.184 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:14.185+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:14.521 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:14.521+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:14.521 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:14.521+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:14.521 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:14.521+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:14.522 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:14.521+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:14.702 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:14.701+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:14.702 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:14.701+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:14.702 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:14.701+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:14.703 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:14.701+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:15.186 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:15.185+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:15.186 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:15.185+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:15.186 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:15.185+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:15.186 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:15.185+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:15.218 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:15.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:15.218 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:15.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:15.218 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:15.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:15.218 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:15.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:15.218 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:15.217+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:15.473 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:15.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:15.473 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:15.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:15.473 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:15.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:15.473 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:15.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:15.696 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:15.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:15.697 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:15.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:15.697 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:15.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:15.697 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:15.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:16.149 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:16.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:16.150 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:16.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:16.150 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:16.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:16.150 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:16.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:16.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:16.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:16.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:16.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:16.179 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:16.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:16.179 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:16.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:16.179 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:16.177+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:16.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:16.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:16.466 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:16.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:16.466 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:16.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:16.466 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:16.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:16.723 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:16.725+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:16.723 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:16.725+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:16.724 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:16.725+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:16.724 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:16.725+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:17.120 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:17.121+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:17.121 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:17.121+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:17.121 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:17.121+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:17.121 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:17.121+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:17.164 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:17.165+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:17.165 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:17.165+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:17.165 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:17.165+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:17.165 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:17.165+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:17.165 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:17.165+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:17.495 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:17.497+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:17.495 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:17.497+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:17.495 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:17.497+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:17.496 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:17.497+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:17.739 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:17.741+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:17.740 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:17.741+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:17.740 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:17.741+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:17.740 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:17.741+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:18.073 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:18.073+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:18.073 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:18.073+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:18.073 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:18.073+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:18.074 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:18.073+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:18.170 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:18.173+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:18.171 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:18.173+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:18.171 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:18.173+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:18.171 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:18.173+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:18.171 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:18.173+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:18.349 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:59:18.349+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 864 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:59:18.464 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:18.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:18.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:18.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:18.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:18.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:18.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:18.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:18.710 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:18.713+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:18.710 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:18.713+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:18.711 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:18.713+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:18.711 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:18.713+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:19.045 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:19.045+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:19.045 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:19.045+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:19.045 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:19.045+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:19.046 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:19.045+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:19.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:19.137+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:19.136 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:19.137+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:19.136 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:19.137+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:19.136 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:19.137+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:19.136 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:19.137+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:19.428 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:19.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:19.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:19.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:19.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:19.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:19.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:19.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:19.685 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:19.685+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:19.685 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:19.685+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:19.685 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:19.685+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:19.686 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:19.685+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:20.024 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:20.025+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:20.024 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:20.025+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:20.024 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:20.025+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:20.025 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:20.025+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:20.154 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:20.153+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:20.154 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:20.153+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:20.155 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:20.153+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:20.155 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:20.153+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:20.155 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:20.153+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:20.460 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:20.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:20.461 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:20.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:20.461 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:20.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:20.461 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:20.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:20.654 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:20.657+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:20.655 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:20.657+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:20.655 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:20.657+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:20.655 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:20.657+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:21.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:21.065+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:21.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:21.065+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:21.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:21.065+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:21.067 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:21.065+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:21.124 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:21.125+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:21.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:21.125+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:21.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:21.125+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:21.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:21.125+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:21.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:21.125+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:21.415 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:21.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:21.416 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:21.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:21.416 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:21.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:21.416 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:21.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:21.678 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:21.677+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:21.679 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:21.677+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:21.679 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:21.677+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:21.679 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:21.677+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:22.026 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:22.029+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:22.027 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:22.029+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:22.027 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:22.029+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:22.027 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:22.029+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:22.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:22.109+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:22.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:22.109+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:22.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:22.109+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:22.108 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:22.109+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:22.108 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:22.109+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:22.445 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:22.445+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:22.446 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:22.445+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:22.446 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:22.445+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:22.446 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:22.445+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:22.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:22.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:22.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:22.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:22.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:22.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:22.672 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:22.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:22.999 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:23.001+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:22.999 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:23.001+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:22.999 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:23.001+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:23.000 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:23.001+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:23.146 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:23.145+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:23.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:23.145+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:23.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:23.145+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:23.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:23.145+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:23.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:23.145+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:23.350 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:59:23.349+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 894 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:59:23.430 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:23.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:23.430 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:23.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:23.431 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:23.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:23.431 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:23.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:23.655 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:23.657+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:23.656 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:23.657+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:23.656 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:23.657+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:23.656 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:23.657+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:24.021 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:24.021+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:24.021 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:24.021+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:24.021 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:24.021+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:24.022 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:24.021+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:24.114 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:24.113+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:24.115 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:24.113+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:24.115 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:24.113+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:24.115 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:24.113+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:24.115 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:24.117+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:24.450 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:24.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:24.451 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:24.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:24.451 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:24.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:24.451 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:24.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:24.695 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:24.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:24.696 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:24.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:24.696 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:24.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:24.696 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:24.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:25.061 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:25.061+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:25.061 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:25.061+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:25.061 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:25.061+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:25.062 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:25.061+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:25.088 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:25.089+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:25.088 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:25.089+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:25.088 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:25.089+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:25.088 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:25.089+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:25.089 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:25.089+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:25.475 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:25.477+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:25.475 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:25.477+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:25.476 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:25.477+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:25.476 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:25.477+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:25.675 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:25.677+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:25.676 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:25.677+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:25.676 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:25.677+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:25.676 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:25.677+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:26.016 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:26.017+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:26.016 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:26.017+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:26.016 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:26.017+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:26.017 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:26.017+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:26.059 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:26.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:26.059 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:26.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:26.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:26.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:26.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:26.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:26.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:26.061+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:26.516 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:26.517+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:26.516 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:26.517+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:26.516 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:26.517+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:26.516 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:26.517+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:26.721 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:26.721+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:26.721 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:26.721+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:26.722 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:26.721+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:26.722 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:26.721+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:27.012 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:27.013+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:27.012 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:27.013+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:27.013 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:27.013+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:27.013 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:27.013+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:27.108 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:27.109+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:27.109 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:27.109+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:27.109 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:27.109+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:27.109 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:27.109+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:27.109 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:27.109+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:27.487 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:27.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:27.488 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:27.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:27.488 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:27.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:27.488 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:27.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:27.679 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:27.681+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:27.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:27.681+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:27.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:27.681+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:27.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:27.681+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:28.054 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:28.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:28.054 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:28.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:28.054 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:28.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:28.054 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:28.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:28.074 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:28.077+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:28.075 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:28.077+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:28.075 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:28.077+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:28.075 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:28.077+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:28.075 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:28.077+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:28.351 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:59:28.353+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 923 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:59:28.442 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:28.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:28.443 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:28.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:28.443 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:28.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:28.443 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:28.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:28.645 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:28.645+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:28.646 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:28.645+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:28.646 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:28.645+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:28.646 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:28.645+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:29.044 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:29.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:29.045 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:29.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:29.045 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:29.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:29.045 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:29.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:29.045 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:29.045+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:29.084 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:29.085+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:29.085 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:29.085+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:29.085 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:29.085+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:29.085 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:29.085+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:29.462 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:29.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:29.462 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:29.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:29.462 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:29.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:29.462 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:29.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:29.695 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:29.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:29.695 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:29.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:29.696 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:29.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:29.696 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:29.697+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:30.011 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:30.013+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:30.011 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:30.013+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:30.011 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:30.013+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:30.012 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:30.013+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:30.012 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:30.013+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:30.127 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:30.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:30.127 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:30.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:30.127 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:30.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:30.127 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:30.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:30.506 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:30.505+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:30.506 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:30.505+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:30.506 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:30.505+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:30.506 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:30.505+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:30.725 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:30.725+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:30.725 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:30.725+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:30.725 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:30.725+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:30.726 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:30.725+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:30.996 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:30.997+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:30.997 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:30.997+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:30.997 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:30.997+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:30.997 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:30.997+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:30.997 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:30.997+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:31.114 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:31.113+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:31.115 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:31.113+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:31.115 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:31.113+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:31.115 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:31.113+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:31.460 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:31.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:31.460 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:31.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:31.461 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:31.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:31.461 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:31.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:31.717 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:31.717+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:31.718 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:31.717+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:31.718 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:31.717+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:31.718 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:31.717+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:32.013 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:32.013+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:32.013 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:32.013+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:32.013 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:32.013+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:32.014 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:32.013+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:32.014 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:32.013+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:32.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:32.161+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:32.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:32.161+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:32.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:32.161+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:32.160 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:32.161+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:32.473 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:32.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:32.474 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:32.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:32.474 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:32.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:32.474 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:32.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:32.735 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:32.737+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:32.735 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:32.737+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:32.735 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:32.737+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:32.736 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:32.737+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:33.046 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:33.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:33.047 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:33.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:33.047 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:33.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:33.047 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:33.045+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:33.047 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:33.049+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:33.146 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:33.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:33.146 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:33.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:33.147 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:33.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:33.147 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:33.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:33.352 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:59:33.353+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 953 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:59:33.468 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:33.469+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:33.468 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:33.469+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:33.468 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:33.469+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:33.469 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:33.469+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:33.740 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:33.741+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:33.741 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:33.741+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:33.741 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:33.741+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:33.741 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:33.741+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:34.054 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:34.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:34.054 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:34.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:34.055 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:34.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:34.055 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:34.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:34.055 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:34.057+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:34.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:34.157+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:34.157 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:34.157+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:34.157 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:34.157+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:34.157 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:34.157+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:34.451 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:34.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:34.452 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:34.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:34.452 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:34.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:34.452 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:34.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:34.782 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:34.781+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:34.783 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:34.781+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:34.783 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:34.781+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:34.783 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:34.781+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:35.074 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:35.073+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:35.075 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:35.073+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:35.075 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:35.073+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:35.075 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:35.073+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:35.075 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:35.073+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:35.188 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:35.189+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:35.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:35.189+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:35.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:35.189+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:35.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:35.189+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:35.428 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:35.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:35.428 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:35.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:35.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:35.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:35.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:35.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:35.775 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:35.777+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:35.776 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:35.777+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:35.776 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:35.777+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:35.776 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:35.777+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:36.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:36.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:36.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:36.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:36.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:36.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:36.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:36.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:36.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:36.105+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:36.169 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:36.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:36.169 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:36.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:36.169 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:36.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:36.169 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:36.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:36.409 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:36.409+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:36.409 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:36.409+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:36.410 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:36.409+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:36.410 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:36.409+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:36.821 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:36.821+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:36.821 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:36.821+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:36.821 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:36.821+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:36.822 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:36.821+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:37.088 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:37.089+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:37.088 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:37.089+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:37.088 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:37.089+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:37.089 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:37.089+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:37.089 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:37.089+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:37.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:37.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:37.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:37.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:37.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:37.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:37.167 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:37.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:37.451 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:37.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:37.451 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:37.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:37.452 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:37.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:37.452 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:37.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:37.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:37.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:37.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:37.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:37.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:37.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:37.869 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:37.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:38.081 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:38.081+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:38.082 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:38.081+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:38.082 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:38.081+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:38.082 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:38.081+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:38.082 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:38.081+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:38.214 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:38.213+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:38.214 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:38.213+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:38.215 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:38.213+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:38.215 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:38.213+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:38.353 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:59:38.353+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 983 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:59:38.449 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:38.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:38.450 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:38.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:38.450 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:38.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:38.450 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:38.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:38.887 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:38.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:38.887 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:38.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:38.887 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:38.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:38.888 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:38.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:39.059 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:39.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:39.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:39.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:39.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:39.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:39.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:39.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:39.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:39.061+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:39.252 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:39.253+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:39.252 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:39.253+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:39.252 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:39.253+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:39.253 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:39.253+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:39.424 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:39.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:39.425 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:39.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:39.425 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:39.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:39.425 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:39.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:39.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:39.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:39.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:39.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:39.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:39.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:39.863 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:39.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:40.062 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:40.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:40.063 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:40.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:40.063 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:40.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:40.063 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:40.061+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:40.063 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:40.061+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:40.226 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:40.225+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:40.226 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:40.225+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:40.227 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:40.225+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:40.227 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:40.225+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:40.453 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:40.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:40.454 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:40.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:40.454 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:40.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:40.454 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:40.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:40.853 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:40.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:40.853 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:40.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:40.853 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:40.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:40.854 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:40.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:41.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:41.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:41.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:41.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:41.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:41.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:41.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:41.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:41.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:41.105+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:41.217 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:41.217+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:41.218 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:41.217+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:41.218 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:41.217+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:41.218 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:41.217+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:41.458 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:41.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:41.458 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:41.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:41.459 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:41.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:41.459 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:41.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:41.839 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:41.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:41.840 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:41.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:41.840 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:41.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:41.840 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:41.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:42.092 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:42.093+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:42.093 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:42.093+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:42.093 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:42.093+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:42.093 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:42.093+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:42.093 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:42.093+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:42.212 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:42.213+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:42.213 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:42.213+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:42.213 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:42.213+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:42.213 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:42.213+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:42.413 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:42.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:42.414 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:42.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:42.414 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:42.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:42.414 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:42.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:42.888 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:42.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:42.889 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:42.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:42.889 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:42.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:42.889 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:42.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:43.089 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:43.089+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:43.090 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:43.089+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:43.090 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:43.089+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:43.090 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:43.089+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:43.090 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:43.089+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:43.228 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:43.229+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:43.228 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:43.229+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:43.228 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:43.229+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:43.228 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:43.229+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:43.354 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:59:43.353+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1019 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:59:43.371 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:43.373+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:43.372 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:43.373+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:43.372 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:43.373+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:43.372 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:43.373+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:43.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:43.873+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:43.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:43.873+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:43.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:43.873+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:43.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:43.873+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:44.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:44.121+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:44.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:44.121+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:44.123 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:44.121+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:44.123 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:44.121+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:44.123 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:44.121+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:44.266 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:44.265+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:44.266 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:44.265+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:44.266 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:44.265+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:44.267 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:44.265+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:44.405 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:44.405+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:44.405 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:44.405+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:44.406 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:44.405+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:44.406 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:44.405+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:44.840 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:44.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:44.840 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:44.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:44.840 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:44.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:44.841 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:44.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:45.126 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:45.129+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:45.126 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:45.129+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:45.126 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:45.129+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:45.127 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:45.129+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:45.127 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:45.129+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:45.244 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:45.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:45.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:45.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:45.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:45.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:45.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:45.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:45.449 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:45.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:45.450 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:45.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:45.450 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:45.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:45.450 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:45.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:45.843 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:45.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:45.843 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:45.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:45.844 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:45.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:45.844 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:45.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:46.136 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:46.137+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:46.136 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:46.137+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:46.136 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:46.137+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:46.137 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:46.137+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:46.137 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:46.137+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:46.275 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:46.277+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:46.275 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:46.277+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:46.275 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:46.277+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:46.276 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:46.277+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:46.461 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:46.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:46.461 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:46.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:46.462 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:46.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:46.462 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:46.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:46.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:46.857+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:46.856 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:46.857+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:46.856 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:46.857+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:46.856 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:46.857+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:47.137 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:47.137+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:47.137 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:47.137+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:47.137 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:47.137+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:47.137 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:47.137+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:47.138 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:47.137+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:47.318 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:47.321+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:47.319 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:47.321+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:47.319 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:47.321+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:47.319 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:47.321+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:47.502 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:47.505+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:47.502 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:47.505+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:47.503 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:47.505+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:47.503 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:47.505+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:47.878 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:47.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:47.879 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:47.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:47.879 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:47.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:47.879 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:47.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:48.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:48.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:48.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:48.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:48.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:48.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:48.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:48.105+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:48.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:48.105+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:48.289 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:48.289+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:48.290 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:48.289+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:48.290 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:48.289+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:48.290 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:48.289+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:48.355 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:59:48.353+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1054 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:59:48.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:48.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:48.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:48.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:48.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:48.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:48.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:48.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:48.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:48.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:48.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:48.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:48.877 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:48.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:48.877 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:48.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:49.149 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:49.149+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:49.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:49.149+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:49.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:49.149+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:49.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:49.149+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:49.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:49.149+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:49.323 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:49.325+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:49.323 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:49.325+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:49.323 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:49.325+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:49.324 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:49.325+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:49.490 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:49.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:49.490 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:49.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:49.490 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:49.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:49.490 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:49.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:49.867 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:49.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:49.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:49.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:49.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:49.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:49.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:49.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:50.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:50.149+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:50.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:50.149+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:50.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:50.149+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:50.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:50.149+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:50.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:50.149+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:50.324 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:50.325+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:50.324 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:50.325+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:50.325 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:50.325+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:50.325 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:50.325+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:50.477 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:50.477+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:50.478 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:50.477+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:50.478 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:50.477+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:50.478 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:50.477+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:50.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:50.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:50.869 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:50.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:50.869 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:50.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:50.869 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:50.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:51.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:51.121+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:51.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:51.121+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:51.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:51.121+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:51.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:51.121+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:51.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:51.121+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:51.341 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:51.345+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:51.342 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:51.345+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:51.342 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:51.345+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:51.342 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:51.345+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:51.462 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:51.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:51.463 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:51.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:51.463 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:51.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:51.464 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:51.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:51.900 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:51.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:51.900 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:51.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:51.901 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:51.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:51.901 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:51.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:52.143 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:52.145+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:52.143 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:52.145+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:52.143 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:52.145+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:52.143 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:52.145+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:52.144 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:52.145+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:52.366 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:52.365+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:52.366 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:52.365+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:52.366 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:52.365+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:52.366 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:52.365+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:52.492 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:52.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:52.492 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:52.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:52.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:52.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:52.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:52.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:52.924 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:52.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:52.924 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:52.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:52.924 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:52.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:52.924 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:52.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:53.131 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:53.133+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:53.131 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:53.133+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:53.131 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:53.133+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:53.131 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:53.133+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:53.132 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:53.133+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:53.355 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:59:53.357+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1089 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:59:53.376 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:53.377+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:53.376 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:53.377+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:53.377 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:53.377+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:53.377 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:53.377+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:53.464 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:53.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:53.464 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:53.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:53.464 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:53.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:53.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:53.465+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:53.961 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:53.961+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:53.961 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:53.961+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:53.961 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:53.961+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:53.962 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:53.961+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:54.157 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:54.157+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:54.157 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:54.157+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:54.157 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:54.157+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:54.158 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:54.157+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:54.158 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:54.157+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:54.399 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:54.401+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:54.399 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:54.401+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:54.399 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:54.401+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:54.400 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:54.401+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:54.509 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:54.509+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:54.509 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:54.509+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:54.510 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:54.509+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:54.510 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:54.509+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:54.993 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:54.993+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:54.993 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:54.993+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:54.994 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:54.993+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:54.994 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:54.993+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:55.199 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:55.201+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:55.200 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:55.201+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:55.200 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:55.201+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:55.200 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:55.201+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:55.200 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:55.201+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:55.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:55.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:55.442 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:55.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:55.442 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:55.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:55.442 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:55.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:55.545 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:55.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:55.546 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:55.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:55.546 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:55.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:55.546 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:55.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:55.984 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:55.984+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:55.985 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:55.984+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:55.985 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:55.984+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:55.985 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:55.984+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:56.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:56.236+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:56.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:56.236+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:56.236 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:56.236+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:56.236 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:56.236+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:56.236 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:56.236+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:56.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:56.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:56.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:56.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:56.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:56.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:56.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:56.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:56.553 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:56.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:56.554 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:56.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:56.554 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:56.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:56.554 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:56.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:56.978 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:56.976+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:56.978 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:56.976+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:56.978 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:56.976+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:56.978 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:56.976+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:57.197 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:57.196+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:57.197 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:57.196+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:57.198 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:57.196+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:57.198 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:57.196+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:57.198 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:57.196+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:57.407 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:57.408+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:57.407 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:57.408+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:57.407 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:57.408+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:57.408 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:57.408+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:57.535 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:57.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:57.535 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:57.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:57.535 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:57.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:57.536 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:57.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:57.981 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:57.980+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:57.981 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:57.980+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:57.982 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:57.980+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:57.982 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:57.980+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:58.242 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:58.240+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:58.243 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:58.240+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:58.243 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:58.240+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:58.243 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:58.240+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:58.243 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:58.240+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:58.356 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T21:59:58.356+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1122 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T21:59:58.439 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:58.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:58.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:58.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:58.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:58.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:58.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:58.440+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:58.585 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:58.584+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:58.586 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:58.584+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:58.586 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:58.584+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:58.586 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:58.584+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:58.957 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:58.956+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:58.957 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:58.956+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:58.957 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:58.956+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:58.957 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:58.956+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:59.260 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:59.260+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:59.260 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:59.260+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:59.260 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:59.260+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:59.260 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:59.260+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T21:59:59.261 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T21:59:59.260+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T21:59:59.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:59.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:59.396 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:59.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:59.396 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:59.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:59.396 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T21:59:59.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T21:59:59.608 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:59.608+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:59.608 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:59.608+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:59.608 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:59.608+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:59.608 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T21:59:59.608+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T21:59:59.907 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:59.908+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:59.908 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:59.908+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:59.908 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:59.908+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T21:59:59.908 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T21:59:59.908+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:00.278 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:00.280+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:00.278 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:00.280+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:00.278 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:00.280+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:00.279 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:00.280+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:00.279 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:00.280+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:00.384 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:00.384+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:00.385 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:00.384+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:00.385 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:00.384+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:00.385 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:00.384+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:00.570 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:00.572+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:00.571 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:00.572+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:00.571 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:00.572+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:00.571 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:00.572+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:00.894 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:00.896+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:00.895 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:00.896+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:00.895 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:00.896+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:00.895 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:00.896+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:01.303 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:01.304+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:01.304 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:01.304+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:01.304 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:01.304+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:01.304 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:01.304+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:01.304 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:01.304+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:01.399 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:01.400+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:01.399 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:01.400+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:01.399 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:01.400+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:01.399 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:01.400+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:01.593 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:01.592+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:01.593 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:01.592+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:01.593 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:01.592+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:01.594 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:01.592+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:01.909 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:01.908+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:01.909 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:01.908+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:01.910 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:01.908+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:01.910 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:01.908+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:02.338 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:02.336+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:02.338 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:02.336+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:02.339 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:02.336+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:02.339 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:02.336+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:02.339 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:02.336+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:02.356 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:02.356+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:02.357 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:02.356+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:02.357 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:02.356+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:02.357 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:02.356+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:02.625 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:02.624+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:02.625 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:02.624+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:02.625 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:02.624+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:02.626 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:02.624+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:02.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:02.860+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:02.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:02.860+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:02.863 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:02.860+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:02.863 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:02.860+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:03.294 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:03.296+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:03.295 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:03.296+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:03.295 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:03.296+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:03.295 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:03.296+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:03.295 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:03.296+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:03.357 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:00:03.356+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1157 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:00:03.364 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:03.364+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:03.364 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:03.364+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:03.365 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:03.364+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:03.365 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:03.364+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:03.586 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:03.588+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:03.587 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:03.588+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:03.587 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:03.588+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:03.587 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:03.588+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:03.833 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:03.832+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:03.834 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:03.832+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:03.834 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:03.832+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:03.834 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:03.832+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:04.258 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:04.260+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:04.259 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:04.260+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:04.259 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:04.260+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:04.259 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:04.260+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:04.259 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:04.260+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:04.322 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:04.320+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:04.322 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:04.320+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:04.323 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:04.320+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:04.323 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:04.320+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:04.546 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:04.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:04.547 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:04.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:04.547 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:04.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:04.547 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:04.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:04.840 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:04.840+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:04.840 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:04.840+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:04.840 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:04.840+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:04.841 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:04.840+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:05.236 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:05.236+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:05.236 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:05.236+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:05.237 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:05.236+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:05.237 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:05.236+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:05.237 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:05.236+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:05.351 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:05.352+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:05.351 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:05.352+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:05.352 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:05.352+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:05.352 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:05.352+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:05.539 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:05.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:05.540 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:05.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:05.540 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:05.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:05.540 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:05.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:05.884 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:05.884+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:05.884 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:05.884+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:05.884 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:05.884+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:05.884 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:05.884+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:06.205 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:06.204+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:06.206 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:06.204+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:06.206 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:06.204+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:06.206 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:06.204+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:06.206 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:06.204+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:06.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:06.388+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:06.390 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:06.388+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:06.390 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:06.388+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:06.390 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:06.388+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:06.534 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:06.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:06.535 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:06.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:06.535 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:06.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:06.535 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:06.536+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:06.865 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:06.864+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:06.866 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:06.864+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:06.866 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:06.864+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:06.866 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:06.864+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:07.188 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:07.188+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:07.188 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:07.188+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:07.188 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:07.188+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:07.189 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:07.188+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:07.189 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:07.188+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:07.361 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:07.360+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:07.361 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:07.360+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:07.362 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:07.360+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:07.362 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:07.360+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:07.554 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:07.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:07.554 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:07.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:07.554 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:07.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:07.554 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:07.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:07.861 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:07.860+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:07.861 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:07.860+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:07.861 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:07.860+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:07.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:07.860+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:08.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:08.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:08.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:08.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:08.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:08.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:08.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:08.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:08.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:08.148+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:08.358 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:00:08.356+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1191 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:00:08.381 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:08.380+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:08.381 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:08.380+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:08.381 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:08.380+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:08.381 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:08.380+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:08.564 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:08.564+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:08.565 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:08.564+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:08.565 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:08.564+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:08.565 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:08.564+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:08.869 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:08.868+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:08.870 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:08.868+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:08.870 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:08.868+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:08.870 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:08.868+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:09.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:09.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:09.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:09.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:09.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:09.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:09.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:09.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:09.149 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:09.148+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:09.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:09.336+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:09.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:09.336+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:09.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:09.336+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:09.337 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:09.336+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:09.515 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:09.516+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:09.516 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:09.516+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:09.516 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:09.516+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:09.516 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:09.516+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:09.822 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:09.820+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:09.822 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:09.820+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:09.823 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:09.820+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:09.823 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:09.820+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:10.146 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:10.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:10.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:10.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:10.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:10.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:10.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:10.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:10.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:10.148+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:10.332 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:10.332+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:10.333 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:10.332+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:10.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:10.332+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:10.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:10.332+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:10.539 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:10.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:10.540 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:10.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:10.540 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:10.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:10.540 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:10.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:10.806 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:10.808+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:10.807 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:10.808+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:10.807 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:10.808+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:10.807 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:10.808+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:11.124 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:11.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:11.124 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:11.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:11.124 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:11.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:11.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:11.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:11.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:11.124+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:11.371 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:11.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:11.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:11.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:11.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:11.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:11.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:11.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:11.539 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:11.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:11.539 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:11.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:11.540 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:11.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:11.540 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:11.540+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:11.820 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:11.820+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:11.821 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:11.820+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:11.821 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:11.820+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:11.821 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:11.820+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:12.153 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:12.152+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:12.154 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:12.152+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:12.154 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:12.152+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:12.154 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:12.152+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:12.154 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:12.152+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:12.391 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:12.392+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:12.391 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:12.392+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:12.391 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:12.392+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:12.391 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:12.392+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:12.494 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:12.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:12.494 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:12.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:12.494 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:12.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:12.494 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:12.492+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:12.795 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:12.796+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:12.796 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:12.796+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:12.796 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:12.796+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:12.796 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:12.796+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:13.177 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:13.180+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:13.177 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:13.180+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:13.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:13.180+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:13.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:13.180+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:13.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:13.180+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:13.345 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:13.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:13.346 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:13.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:13.346 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:13.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:13.346 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:13.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:13.359 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:00:13.360+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1226 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:00:13.503 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:13.504+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:13.503 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:13.504+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:13.504 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:13.504+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:13.504 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:13.504+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:13.754 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:13.752+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:13.754 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:13.752+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:13.755 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:13.752+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:13.755 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:13.752+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:14.186 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:14.184+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:14.186 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:14.184+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:14.187 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:14.184+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:14.187 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:14.184+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:14.187 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:14.184+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:14.327 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:14.328+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:14.328 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:14.328+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:14.329 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:14.328+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:14.329 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:14.328+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:14.551 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:14.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:14.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:14.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:14.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:14.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:14.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:14.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:14.797 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:14.796+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:14.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:14.796+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:14.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:14.796+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:14.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:14.796+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:15.173 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:15.176+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:15.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:15.176+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:15.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:15.176+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:15.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:15.176+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:15.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:15.176+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:15.315 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:15.316+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:15.315 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:15.316+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:15.315 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:15.316+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:15.316 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:15.316+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:15.549 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:15.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:15.549 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:15.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:15.549 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:15.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:15.549 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:15.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:15.759 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:15.760+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:15.760 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:15.760+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:15.760 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:15.760+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:15.760 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:15.760+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:16.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:16.212+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:16.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:16.212+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:16.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:16.212+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:16.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:16.212+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:16.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:16.212+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:16.331 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:16.332+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:16.332 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:16.332+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:16.332 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:16.332+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:16.332 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:16.332+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:16.506 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:16.504+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:16.506 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:16.504+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:16.507 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:16.504+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:16.507 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:16.504+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:16.797 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:16.796+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:16.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:16.796+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:16.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:16.796+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:16.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:16.796+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:17.251 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:17.252+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:17.251 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:17.252+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:17.251 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:17.252+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:17.252 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:17.252+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:17.252 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:17.252+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:17.338 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:17.336+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:17.338 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:17.336+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:17.338 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:17.336+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:17.339 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:17.336+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:17.466 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:17.468+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:17.466 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:17.468+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:17.467 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:17.468+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:17.467 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:17.468+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:17.822 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:17.824+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:17.822 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:17.824+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:17.822 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:17.824+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:17.823 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:17.824+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:18.210 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:18.208+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:18.211 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:18.208+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:18.211 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:18.208+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:18.211 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:18.208+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:18.211 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:18.208+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:18.358 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:18.356+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:18.359 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:18.356+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:18.359 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:18.356+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:18.359 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:18.356+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:18.361 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:00:18.360+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1261 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:00:18.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:18.484+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:18.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:18.484+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:18.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:18.484+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:18.485 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:18.484+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:18.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:18.868+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:18.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:18.868+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:18.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:18.868+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:18.869 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:18.868+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:19.255 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:19.256+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:19.255 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:19.256+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:19.255 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:19.256+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:19.256 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:19.256+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:19.256 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:19.256+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:19.378 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:19.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:19.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:19.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:19.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:19.380+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:19.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:19.380+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:19.534 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:19.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:19.534 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:19.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:19.534 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:19.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:19.534 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:19.532+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:19.886 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:19.888+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:19.886 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:19.888+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:19.886 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:19.888+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:19.886 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:19.888+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:20.278 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:20.280+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:20.279 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:20.280+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:20.279 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:20.280+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:20.279 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:20.280+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:20.279 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:20.280+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:20.396 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:20.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:20.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:20.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:20.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:20.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:20.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:20.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:20.556 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:20.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:20.557 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:20.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:20.557 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:20.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:20.557 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:20.556+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:20.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:20.864+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:20.865 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:20.864+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:20.865 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:20.864+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:20.865 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:20.864+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:21.282 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:21.284+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:21.283 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:21.284+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:21.283 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:21.284+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:21.283 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:21.284+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:21.283 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:21.284+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:21.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:21.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:21.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:21.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:21.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:21.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:21.373 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:21.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:21.585 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:21.588+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:21.585 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:21.588+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:21.585 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:21.588+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:21.586 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:21.588+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:21.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:21.904+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:21.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:21.904+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:21.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:21.904+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:21.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:21.904+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:22.246 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:22.248+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:22.247 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:22.248+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:22.247 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:22.248+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:22.247 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:22.248+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:22.247 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:22.248+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:22.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:22.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:22.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:22.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:22.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:22.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:22.395 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:22.396+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:22.567 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:22.568+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:22.567 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:22.568+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:22.568 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:22.568+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:22.568 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:22.568+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:22.907 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:22.908+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:22.907 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:22.908+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:22.907 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:22.908+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:22.908 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:22.908+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:23.202 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:23.200+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:23.203 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:23.204+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:23.203 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:23.204+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:23.203 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:23.204+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:23.203 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:23.204+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:23.361 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:00:23.364+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1296 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:00:23.370 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:23.368+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:23.370 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:23.368+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:23.370 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:23.368+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:23.371 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:23.368+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:23.518 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:23.516+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:23.518 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:23.516+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:23.519 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:23.516+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:23.519 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:23.516+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:23.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:23.876+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:23.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:23.876+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:23.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:23.876+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:23.877 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:23.876+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:24.203 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:24.204+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:24.203 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:24.204+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:24.204 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:24.204+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:24.204 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:24.204+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:24.204 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:24.204+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:24.354 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:24.352+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:24.354 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:24.352+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:24.354 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:24.352+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:24.354 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:24.352+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:24.549 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:24.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:24.550 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:24.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:24.550 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:24.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:24.550 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:24.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:24.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:24.856+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:24.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:24.856+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:24.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:24.856+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:24.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:24.856+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:25.196 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:25.196+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:25.196 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:25.196+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:25.196 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:25.196+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:25.197 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:25.196+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:25.197 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:25.196+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:25.318 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:25.316+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:25.318 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:25.316+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:25.318 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:25.316+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:25.319 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:25.316+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:25.548 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:25.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:25.548 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:25.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:25.548 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:25.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:25.549 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:25.548+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:25.890 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:25.892+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:25.890 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:25.892+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:25.890 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:25.892+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:25.891 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:25.892+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:26.168 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:26.168+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:26.169 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:26.168+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:26.169 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:26.168+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:26.169 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:26.168+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:26.169 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:26.168+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:26.290 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:26.292+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:26.290 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:26.292+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:26.290 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:26.292+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:26.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:26.292+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:26.554 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:26.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:26.554 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:26.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:26.554 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:26.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:26.554 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:26.552+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:26.870 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:26.872+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:26.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:26.872+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:26.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:26.872+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:26.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:26.872+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:27.119 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:27.120+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:27.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:27.120+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:27.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:27.120+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:27.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:27.120+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:27.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:27.120+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:27.276 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:27.276+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:27.277 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:27.276+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:27.277 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:27.276+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:27.277 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:27.276+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:27.591 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:27.592+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:27.591 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:27.592+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:27.592 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:27.592+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:27.592 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:27.592+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:27.857 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:27.856+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:27.857 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:27.856+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:27.857 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:27.856+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:27.858 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:27.856+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:28.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:28.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:28.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:28.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:28.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:28.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:28.149 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:28.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:28.149 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:28.148+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:28.269 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:28.268+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:28.269 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:28.268+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:28.269 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:28.268+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:28.269 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:28.268+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:28.362 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:00:28.360+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1329 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:00:28.577 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:28.576+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:28.578 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:28.576+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:28.578 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:28.576+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:28.578 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:28.576+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:28.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:28.892+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:28.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:28.892+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:28.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:28.892+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:28.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:28.892+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:29.196 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:29.196+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:29.196 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:29.196+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:29.196 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:29.196+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:29.196 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:29.196+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:29.197 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:29.196+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:29.242 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:29.240+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:29.243 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:29.240+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:29.243 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:29.240+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:29.243 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:29.244+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:29.566 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:29.564+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:29.566 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:29.564+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:29.567 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:29.564+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:29.567 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:29.564+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:29.879 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:29.880+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:29.879 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:29.880+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:29.880 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:29.880+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:29.880 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:29.880+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:30.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:30.184+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:30.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:30.184+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:30.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:30.184+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:30.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:30.184+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:30.184 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:30.184+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:30.257 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:30.256+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:30.257 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:30.256+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:30.257 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:30.256+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:30.257 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:30.256+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:30.613 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:30.616+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:30.614 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:30.616+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:30.614 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:30.616+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:30.614 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:30.616+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:30.929 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:30.932+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:30.929 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:30.932+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:30.929 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:30.932+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:30.930 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:30.932+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:31.226 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:31.228+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:31.226 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:31.228+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:31.226 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:31.228+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:31.227 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:31.228+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:31.227 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:31.228+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:31.294 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:31.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:31.295 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:31.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:31.295 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:31.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:31.295 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:31.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:31.647 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:31.648+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:31.647 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:31.648+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:31.647 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:31.648+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:31.647 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:31.648+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:31.942 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:31.944+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:31.942 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:31.944+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:31.942 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:31.944+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:31.942 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:31.944+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:32.258 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:32.260+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:32.259 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:32.260+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:32.259 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:32.260+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:32.259 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:32.260+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:32.260 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:32.260+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:32.260 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:32.260+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:32.260 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:32.260+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:32.261 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:32.260+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:32.261 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:32.260+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:32.663 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:32.663+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:32.663 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:32.663+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:32.664 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:32.663+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:32.664 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:32.663+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:32.931 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:32.931+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:32.932 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:32.931+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:32.932 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:32.931+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:32.932 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:32.931+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:33.227 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:33.227+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:33.228 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:33.227+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:33.228 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:33.227+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:33.228 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:33.227+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:33.228 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:33.227+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:33.295 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:33.295+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:33.296 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:33.295+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:33.296 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:33.295+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:33.296 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:33.295+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:33.363 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:00:33.363+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1365 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:00:33.708 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:33.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:33.708 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:33.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:33.708 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:33.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:33.708 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:33.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:33.959 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:33.959+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:33.959 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:33.959+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:33.959 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:33.959+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:33.960 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:33.959+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:34.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:34.235+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:34.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:34.235+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:34.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:34.235+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:34.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:34.235+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:34.236 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:34.235+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:34.252 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:34.251+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:34.253 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:34.251+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:34.253 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:34.251+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:34.253 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:34.251+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:34.678 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:34.679+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:34.679 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:34.679+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:34.679 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:34.679+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:34.679 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:34.679+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:34.988 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:34.987+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:34.988 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:34.987+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:34.989 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:34.987+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:34.989 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:34.987+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:35.278 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:35.275+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:35.278 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:35.275+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:35.279 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:35.275+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:35.279 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:35.275+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:35.283 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:35.283+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:35.283 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:35.283+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:35.283 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:35.283+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:35.283 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:35.283+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:35.284 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:35.283+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:35.703 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:35.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:35.703 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:35.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:35.703 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:35.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:35.704 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:35.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:35.993 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:35.995+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:35.993 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:35.995+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:35.993 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:35.995+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:35.994 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:35.995+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:36.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:36.283+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:36.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:36.283+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:36.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:36.283+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:36.286 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:36.283+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:36.286 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:36.283+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:36.296 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:36.295+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:36.296 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:36.295+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:36.296 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:36.295+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:36.297 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:36.295+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:36.666 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:36.663+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:36.667 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:36.663+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:36.667 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:36.663+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:36.667 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:36.663+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:37.018 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:37.015+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:37.018 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:37.015+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:37.019 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:37.015+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:37.019 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:37.015+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:37.273 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:37.275+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:37.274 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:37.275+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:37.274 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:37.275+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:37.274 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:37.275+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:37.274 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:37.275+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:37.283 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:37.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:37.283 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:37.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:37.284 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:37.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:37.284 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:37.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:37.658 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:37.655+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:37.658 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:37.655+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:37.659 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:37.655+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:37.659 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:37.655+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:38.040 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:38.039+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:38.040 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:38.039+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:38.040 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:38.039+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:38.041 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:38.039+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:38.283 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:38.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:38.283 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:38.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:38.284 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:38.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:38.284 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:38.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:38.322 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:38.319+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:38.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:38.319+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:38.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:38.319+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:38.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:38.319+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:38.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:38.323+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:38.364 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:00:38.363+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1399 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:00:38.673 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:38.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:38.674 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:38.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:38.674 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:38.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:38.674 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:38.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:39.073 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:39.075+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:39.074 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:39.075+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:39.074 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:39.075+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:39.074 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:39.075+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:39.247 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:39.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:39.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:39.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:39.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:39.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:39.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:39.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:39.357 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:39.359+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:39.358 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:39.359+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:39.358 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:39.359+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:39.358 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:39.359+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:39.358 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:39.359+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:39.668 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:39.667+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:39.668 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:39.667+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:39.668 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:39.667+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:39.669 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:39.667+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:40.026 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:40.023+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:40.027 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:40.023+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:40.027 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:40.023+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:40.027 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:40.023+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:40.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:40.291+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:40.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:40.291+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:40.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:40.291+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:40.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:40.291+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:40.352 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:40.355+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:40.353 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:40.355+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:40.353 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:40.355+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:40.353 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:40.355+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:40.353 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:40.355+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:40.639 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:40.639+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:40.639 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:40.639+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:40.639 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:40.639+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:40.639 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:40.639+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:40.990 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:40.987+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:40.990 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:40.987+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:40.991 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:40.987+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:40.991 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:40.987+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:41.251 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:41.251+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:41.252 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:41.251+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:41.252 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:41.251+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:41.252 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:41.251+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:41.350 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:41.351+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:41.351 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:41.351+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:41.351 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:41.351+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:41.351 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:41.351+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:41.351 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:41.351+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:41.596 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:41.595+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:41.596 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:41.595+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:41.596 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:41.595+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:41.597 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:41.595+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:41.966 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:41.967+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:41.966 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:41.967+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:41.967 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:41.967+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:41.967 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:41.967+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:42.241 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:42.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:42.242 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:42.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:42.242 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:42.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:42.242 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:42.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:42.316 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:42.315+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:42.316 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:42.315+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:42.317 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:42.315+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:42.317 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:42.315+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:42.317 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:42.315+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:42.589 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:42.591+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:42.590 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:42.591+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:42.590 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:42.591+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:42.590 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:42.591+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:42.999 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:42.999+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:42.999 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:42.999+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:42.999 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:42.999+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:43.000 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:42.999+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:43.280 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:43.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:43.281 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:43.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:43.281 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:43.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:43.281 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:43.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:43.341 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:43.339+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:43.341 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:43.339+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:43.341 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:43.339+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:43.342 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:43.339+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:43.342 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:43.339+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:43.365 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:00:43.367+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1435 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:00:43.640 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:43.639+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:43.640 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:43.639+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:43.640 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:43.639+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:43.640 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:43.639+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:44.003 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:44.003+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:44.004 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:44.003+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:44.004 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:44.003+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:44.004 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:44.003+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:44.232 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:44.231+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:44.232 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:44.231+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:44.233 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:44.231+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:44.233 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:44.231+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:44.294 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:44.295+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:44.295 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:44.295+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:44.295 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:44.295+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:44.295 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:44.295+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:44.295 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:44.295+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:44.657 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:44.659+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:44.657 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:44.659+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:44.657 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:44.659+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:44.658 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:44.659+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:45.051 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:45.051+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:45.051 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:45.051+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:45.051 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:45.051+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:45.052 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:45.051+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:45.187 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:45.187+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:45.187 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:45.187+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:45.187 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:45.187+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:45.187 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:45.187+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:45.314 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:45.315+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:45.315 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:45.315+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:45.315 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:45.315+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:45.315 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:45.315+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:45.315 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:45.315+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:45.676 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:45.679+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:45.677 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:45.679+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:45.677 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:45.679+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:45.677 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:45.679+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:46.042 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:46.039+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:46.043 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:46.039+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:46.043 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:46.039+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:46.043 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:46.039+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:46.150 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:46.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:46.150 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:46.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:46.151 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:46.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:46.151 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:46.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:46.268 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:46.271+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:46.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:46.271+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:46.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:46.271+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:46.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:46.271+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:46.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:46.271+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:46.722 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:46.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:46.722 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:46.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:46.722 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:46.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:46.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:46.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:47.017 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:47.019+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:47.017 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:47.019+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:47.018 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:47.019+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:47.018 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:47.019+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:47.101 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:47.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:47.101 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:47.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:47.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:47.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:47.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:47.103+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:47.306 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:47.307+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:47.307 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:47.307+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:47.307 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:47.307+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:47.307 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:47.307+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:47.307 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:47.307+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:47.687 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:47.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:47.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:47.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:47.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:47.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:47.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:47.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:48.025 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:48.027+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:48.026 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:48.027+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:48.026 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:48.027+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:48.026 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:48.027+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:48.098 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:48.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:48.098 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:48.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:48.098 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:48.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:48.099 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:48.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:48.335 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:48.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:48.335 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:48.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:48.336 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:48.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:48.336 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:48.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:48.336 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:48.335+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:48.367 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:00:48.367+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1468 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:00:48.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:48.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:48.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:48.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:48.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:48.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:48.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:48.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:49.070 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:49.067+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:49.071 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:49.067+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:49.071 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:49.067+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:49.071 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:49.067+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:49.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:49.099+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:49.103 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:49.099+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:49.103 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:49.099+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:49.103 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:49.099+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:49.364 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:49.363+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:49.364 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:49.363+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:49.364 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:49.363+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:49.365 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:49.363+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:49.365 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:49.363+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:49.698 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:49.695+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:49.699 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:49.695+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:49.699 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:49.695+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:49.699 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:49.695+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:50.104 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:50.103+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:50.105 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:50.103+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:50.105 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:50.103+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:50.105 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:50.103+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:50.114 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:50.115+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:50.114 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:50.115+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:50.114 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:50.115+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:50.114 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:50.115+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:50.348 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:50.347+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:50.348 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:50.347+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:50.349 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:50.347+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:50.349 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:50.347+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:50.349 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:50.347+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:50.690 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:50.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:50.691 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:50.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:50.691 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:50.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:50.691 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:50.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:51.065 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:51.063+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:51.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:51.063+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:51.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:51.063+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:51.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:51.063+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:51.147 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:51.147+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:51.147 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:51.147+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:51.147 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:51.147+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:51.148 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:51.147+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:51.375 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:51.375+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:51.376 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:51.375+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:51.376 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:51.375+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:51.376 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:51.375+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:51.376 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:51.375+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:51.721 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:51.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:51.721 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:51.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:51.722 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:51.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:51.722 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:51.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:52.065 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:52.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:52.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:52.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:52.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:52.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:52.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:52.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:52.114 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:52.111+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:52.114 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:52.111+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:52.114 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:52.111+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:52.114 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:52.111+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:52.401 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:52.399+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:52.402 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:52.399+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:52.402 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:52.399+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:52.402 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:52.399+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:52.402 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:52.399+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:52.708 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:52.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:52.709 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:52.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:52.709 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:52.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:52.709 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:52.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:53.026 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:53.023+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:53.026 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:53.023+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:53.027 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:53.023+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:53.027 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:53.023+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:53.160 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:53.159+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:53.160 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:53.159+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:53.160 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:53.159+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:53.160 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:53.159+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:53.367 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:00:53.367+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1504 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:00:53.426 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:53.423+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:53.426 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:53.423+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:53.426 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:53.423+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:53.426 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:53.423+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:53.427 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:53.423+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:53.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:53.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:53.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:53.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:53.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:53.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:53.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:53.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:54.033 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:54.031+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:54.033 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:54.031+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:54.033 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:54.031+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:54.034 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:54.031+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:54.118 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:54.119+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:54.118 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:54.119+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:54.119 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:54.119+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:54.119 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:54.119+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:54.419 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:54.419+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:54.420 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:54.419+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:54.420 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:54.419+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:54.420 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:54.419+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:54.420 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:54.419+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:54.719 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:54.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:54.720 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:54.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:54.720 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:54.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:54.720 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:54.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:55.072 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:55.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:55.073 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:55.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:55.073 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:55.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:55.073 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:55.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:55.128 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:55.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:55.129 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:55.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:55.129 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:55.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:55.129 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:55.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:55.453 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:55.455+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:55.454 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:55.455+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:55.454 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:55.455+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:55.454 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:55.455+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:55.454 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:55.455+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:55.678 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:55.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:55.679 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:55.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:55.679 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:55.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:55.679 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:55.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:56.067 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:56.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:56.067 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:56.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:56.067 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:56.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:56.068 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:56.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:56.103 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:56.103+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:56.103 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:56.103+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:56.104 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:56.103+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:56.104 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:56.103+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:56.461 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:56.463+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:56.461 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:56.463+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:56.461 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:56.463+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:56.461 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:56.463+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:56.462 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:56.463+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:56.650 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:56.651+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:56.650 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:56.651+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:56.650 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:56.651+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:56.650 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:56.651+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:57.083 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:57.083+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:57.084 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:57.083+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:57.084 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:57.083+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:57.084 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:57.083+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:57.152 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:57.155+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:57.152 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:57.155+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:57.153 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:57.155+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:57.153 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:57.155+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:57.417 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:57.415+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:57.417 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:57.415+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:57.418 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:57.415+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:57.418 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:57.415+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:57.418 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:57.415+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:57.646 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:57.647+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:57.646 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:57.647+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:57.647 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:57.647+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:57.647 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:57.647+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:58.108 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:58.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:58.108 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:58.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:58.109 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:58.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:58.109 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:58.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:58.172 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:58.175+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:58.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:58.175+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:58.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:58.175+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:58.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:58.175+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:58.369 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:00:58.367+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1538 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:00:58.439 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:58.439+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:58.440 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:58.439+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:58.440 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:58.439+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:58.440 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:58.439+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:58.440 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:58.439+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:58.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:58.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:58.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:58.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:58.689 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:58.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:58.689 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:58.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:59.145 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:59.147+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:59.145 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:59.147+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:59.145 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:59.147+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:59.146 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:00:59.147+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:00:59.158 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:59.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:59.158 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:59.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:59.158 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:59.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:59.158 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:00:59.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:00:59.455 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:59.455+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:59.455 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:59.455+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:59.456 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:59.455+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:59.456 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:59.455+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:00:59.456 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:00:59.455+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:00:59.690 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:59.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:59.690 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:59.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:59.690 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:59.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:00:59.691 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:00:59.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:00.114 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:00.111+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:00.115 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:00.111+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:00.115 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:00.111+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:00.115 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:00.111+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:00.129 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:00.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:00.129 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:00.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:00.129 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:00.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:00.130 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:00.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:00.438 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:00.435+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:00.438 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:00.435+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:00.438 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:00.435+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:00.439 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:00.435+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:00.439 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:00.435+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:00.713 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:00.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:00.714 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:00.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:00.714 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:00.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:00.714 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:00.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:01.087 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:01.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:01.087 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:01.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:01.088 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:01.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:01.088 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:01.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:01.141 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:01.143+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:01.141 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:01.143+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:01.141 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:01.143+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:01.142 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:01.143+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:01.450 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:01.451+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:01.451 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:01.451+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:01.451 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:01.451+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:01.451 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:01.451+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:01.451 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:01.451+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:01.755 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:01.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:01.755 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:01.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:01.756 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:01.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:01.756 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:01.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:02.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:02.063+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:02.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:02.063+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:02.064 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:02.063+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:02.064 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:02.063+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:02.176 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:02.179+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:02.176 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:02.179+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:02.176 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:02.179+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:02.177 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:02.179+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:02.476 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:02.479+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:02.477 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:02.479+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:02.477 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:02.479+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:02.477 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:02.479+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:02.478 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:02.479+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:02.726 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:02.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:02.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:02.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:02.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:02.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:02.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:02.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:03.048 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:03.051+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:03.049 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:03.051+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:03.049 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:03.051+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:03.049 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:03.051+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:03.217 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:03.215+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:03.217 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:03.215+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:03.218 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:03.215+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:03.218 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:03.215+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:03.370 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:01:03.371+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1574 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:01:03.476 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:03.475+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:03.476 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:03.475+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:03.476 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:03.475+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:03.476 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:03.475+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:03.477 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:03.475+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:03.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:03.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:03.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:03.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:03.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:03.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:03.724 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:03.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:04.078 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:04.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:04.078 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:04.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:04.078 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:04.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:04.079 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:04.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:04.185 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:04.183+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:04.185 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:04.183+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:04.185 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:04.183+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:04.186 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:04.183+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:04.445 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:04.443+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:04.445 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:04.443+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:04.446 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:04.443+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:04.446 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:04.443+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:04.446 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:04.443+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:04.673 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:04.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:04.674 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:04.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:04.674 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:04.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:04.674 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:04.675+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:05.030 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:05.027+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:05.030 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:05.027+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:05.030 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:05.027+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:05.031 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:05.027+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:05.204 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:05.207+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:05.204 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:05.207+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:05.205 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:05.207+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:05.205 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:05.207+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:05.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:05.423+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:05.426 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:05.423+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:05.426 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:05.423+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:05.426 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:05.423+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:05.426 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:05.423+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:05.692 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:05.695+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:05.693 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:05.695+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:05.693 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:05.695+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:05.693 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:05.695+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:06.059 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:06.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:06.059 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:06.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:06.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:06.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:06.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:06.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:06.230 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:06.231+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:06.230 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:06.231+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:06.230 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:06.231+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:06.231 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:06.231+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:06.428 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:06.427+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:06.428 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:06.427+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:06.428 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:06.427+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:06.428 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:06.427+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:06.429 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:06.427+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:06.720 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:06.722+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:06.720 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:06.722+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:06.721 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:06.722+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:06.721 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:06.722+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:07.010 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:07.010+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:07.011 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:07.010+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:07.011 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:07.010+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:07.011 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:07.010+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:07.202 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:07.198+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:07.202 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:07.198+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:07.202 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:07.198+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:07.203 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:07.198+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:07.475 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:07.474+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:07.475 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:07.474+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:07.476 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:07.474+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:07.476 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:07.474+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:07.476 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:07.474+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:07.750 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:07.750+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:07.751 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:07.750+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:07.751 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:07.750+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:07.751 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:07.750+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:07.978 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:07.974+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:07.978 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:07.974+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:07.979 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:07.974+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:07.979 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:07.974+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:08.243 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:08.242+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:08.243 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:08.242+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:08.243 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:08.242+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:08.244 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:08.242+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:08.371 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:01:08.370+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1607 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:01:08.492 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:08.490+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:08.492 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:08.490+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:08.492 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:08.490+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:08.493 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:08.490+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:08.493 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:08.490+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:08.776 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:08.774+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:08.776 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:08.774+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:08.776 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:08.774+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:08.776 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:08.774+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:08.971 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:08.970+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:08.971 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:08.970+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:08.971 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:08.970+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:08.972 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:08.970+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:09.264 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:09.266+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:09.264 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:09.266+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:09.264 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:09.266+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:09.265 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:09.266+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:09.451 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:09.450+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:09.451 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:09.450+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:09.451 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:09.450+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:09.452 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:09.450+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:09.452 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:09.450+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:09.754 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:09.750+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:09.754 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:09.750+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:09.755 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:09.750+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:09.755 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:09.750+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:09.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:09.982+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:09.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:09.982+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:09.987 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:09.982+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:09.987 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:09.982+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:10.263 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:10.262+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:10.263 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:10.262+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:10.264 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:10.262+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:10.264 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:10.262+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:10.476 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:10.474+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:10.477 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:10.474+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:10.477 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:10.474+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:10.477 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:10.474+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:10.477 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:10.474+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:10.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:10.794+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:10.795 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:10.794+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:10.795 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:10.794+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:10.795 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:10.794+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:11.001 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:10.998+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:11.001 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:10.998+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:11.002 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:10.998+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:11.002 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:10.998+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:11.297 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:11.298+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:11.298 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:11.298+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:11.298 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:11.298+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:11.298 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:11.298+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:11.497 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:11.498+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:11.498 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:11.498+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:11.498 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:11.498+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:11.498 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:11.498+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:11.498 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:11.498+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:11.766 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:11.766+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:11.766 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:11.766+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:11.766 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:11.766+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:11.766 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:11.766+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:11.976 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:11.978+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:11.976 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:11.978+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:11.976 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:11.978+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:11.977 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:11.978+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:12.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:12.298+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:12.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:12.298+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:12.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:12.298+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:12.297 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:12.298+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:12.463 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:12.462+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:12.463 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:12.462+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:12.463 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:12.462+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:12.464 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:12.462+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:12.464 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:12.462+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:12.758 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:12.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:12.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:12.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:12.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:12.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:12.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:12.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:12.945 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:12.946+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:12.945 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:12.946+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:12.946 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:12.946+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:12.946 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:12.946+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:13.269 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:13.270+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:13.270 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:13.270+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:13.270 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:13.270+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:13.270 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:13.270+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:13.372 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:01:13.370+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1643 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:01:13.442 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:13.438+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:13.442 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:13.438+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:13.443 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:13.438+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:13.443 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:13.438+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:13.443 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:13.438+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:13.736 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:13.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:13.736 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:13.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:13.737 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:13.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:13.737 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:13.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:13.912 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:13.910+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:13.912 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:13.910+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:13.912 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:13.910+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:13.912 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:13.910+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:14.228 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:14.226+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:14.228 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:14.226+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:14.228 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:14.226+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:14.228 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:14.226+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:14.467 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:14.466+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:14.467 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:14.466+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:14.467 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:14.466+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:14.468 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:14.466+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:14.468 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:14.466+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:14.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:14.726+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:14.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:14.726+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:14.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:14.726+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:14.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:14.726+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:14.922 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:14.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:14.923 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:14.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:14.923 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:14.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:14.923 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:14.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:15.206 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:15.202+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:15.206 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:15.202+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:15.207 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:15.202+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:15.207 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:15.202+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:15.418 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:15.414+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:15.419 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:15.414+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:15.419 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:15.414+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:15.419 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:15.414+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:15.419 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:15.414+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:15.730 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:15.730+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:15.730 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:15.730+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:15.730 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:15.730+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:15.731 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:15.730+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:15.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:15.934+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:15.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:15.934+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:15.939 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:15.934+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:15.939 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:15.934+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:16.180 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:16.182+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:16.180 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:16.182+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:16.180 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:16.182+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:16.180 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:16.182+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:16.398 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:16.398+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:16.398 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:16.398+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:16.398 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:16.398+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:16.398 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:16.398+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:16.399 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:16.398+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:16.755 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:16.754+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:16.755 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:16.754+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:16.755 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:16.754+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:16.755 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:16.754+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:16.985 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:16.982+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:16.985 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:16.982+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:16.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:16.982+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:16.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:16.982+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:17.145 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:17.142+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:17.146 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:17.142+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:17.146 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:17.142+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:17.146 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:17.142+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:17.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:17.382+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:17.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:17.382+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:17.382 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:17.382+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:17.382 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:17.382+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:17.382 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:17.382+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:17.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:17.706+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:17.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:17.706+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:17.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:17.706+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:17.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:17.706+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:17.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:17.982+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:17.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:17.982+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:17.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:17.982+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:17.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:17.982+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:18.171 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:18.170+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:18.171 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:18.170+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:18.171 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:18.170+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:18.172 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:18.170+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:18.373 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:01:18.374+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1677 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:01:18.396 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:18.394+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:18.397 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:18.394+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:18.397 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:18.394+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:18.397 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:18.394+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:18.397 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:18.394+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:18.685 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:18.686+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:18.685 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:18.686+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:18.685 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:18.686+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:18.685 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:18.686+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:19.016 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:19.018+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:19.017 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:19.018+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:19.017 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:19.018+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:19.017 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:19.018+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:19.128 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:19.126+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:19.128 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:19.126+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:19.129 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:19.126+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:19.129 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:19.126+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:19.352 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:19.350+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:19.353 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:19.350+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:19.353 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:19.350+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:19.353 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:19.350+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:19.354 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:19.350+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:19.651 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:19.650+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:19.651 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:19.650+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:19.652 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:19.650+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:19.652 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:19.650+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:19.999 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:19.998+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:20.000 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:19.998+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:20.000 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:19.998+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:20.000 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:19.998+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:20.114 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:20.110+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:20.115 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:20.110+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:20.115 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:20.110+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:20.115 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:20.110+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:20.392 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:20.394+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:20.392 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:20.394+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:20.393 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:20.394+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:20.393 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:20.394+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:20.393 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:20.394+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:20.645 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:20.646+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:20.645 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:20.646+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:20.645 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:20.646+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:20.646 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:20.646+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:20.969 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:20.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:20.970 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:20.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:20.970 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:20.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:20.970 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:20.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:21.121 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:21.122+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:21.121 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:21.122+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:21.121 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:21.122+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:21.122 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:21.122+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:21.360 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:21.358+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:21.360 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:21.358+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:21.361 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:21.358+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:21.361 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:21.358+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:21.361 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:21.358+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:21.620 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:21.622+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:21.620 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:21.622+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:21.620 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:21.622+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:21.620 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:21.622+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:21.940 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:21.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:21.940 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:21.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:21.940 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:21.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:21.940 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:21.938+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:22.170 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:22.166+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:22.171 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:22.166+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:22.171 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:22.166+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:22.171 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:22.166+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:22.322 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:22.322+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:22.322 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:22.322+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:22.322 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:22.322+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:22.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:22.322+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:22.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:22.322+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:22.639 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:22.638+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:22.639 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:22.638+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:22.640 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:22.638+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:22.640 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:22.638+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:22.900 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:22.902+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:22.900 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:22.902+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:22.900 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:22.902+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:22.900 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:22.902+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:23.128 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:23.126+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:23.128 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:23.126+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:23.128 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:23.126+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:23.128 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:23.126+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:23.287 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:23.290+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:23.288 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:23.290+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:23.288 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:23.290+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:23.288 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:23.290+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:23.288 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:23.290+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:23.374 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:01:23.374+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1713 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:01:23.675 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:23.670+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:23.675 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:23.674+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:23.675 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:23.674+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:23.676 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:23.674+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:23.866 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:23.866+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:23.866 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:23.866+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:23.866 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:23.866+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:23.866 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:23.866+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:24.085 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:24.086+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:24.086 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:24.086+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:24.086 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:24.086+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:24.086 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:24.086+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:24.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:24.286+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:24.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:24.286+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:24.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:24.286+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:24.286 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:24.286+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:24.286 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:24.286+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:24.677 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:24.678+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:24.677 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:24.678+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:24.678 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:24.678+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:24.678 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:24.678+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:24.891 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:24.890+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:24.891 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:24.890+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:24.892 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:24.890+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:24.892 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:24.890+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:25.043 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:25.046+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:25.044 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:25.046+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:25.044 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:25.046+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:25.044 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:25.046+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:25.330 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:25.326+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:25.331 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:25.326+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:25.331 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:25.326+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:25.331 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:25.326+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:25.331 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:25.326+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:25.634 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:25.634+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:25.635 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:25.634+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:25.635 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:25.634+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:25.635 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:25.634+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:25.846 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:25.846+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:25.847 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:25.846+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:25.847 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:25.846+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:25.848 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:25.846+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:26.094 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:26.090+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:26.094 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:26.090+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:26.094 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:26.090+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:26.095 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:26.090+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:26.380 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:26.382+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:26.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:26.382+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:26.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:26.382+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:26.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:26.382+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:26.382 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:26.382+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:26.593 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:26.594+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:26.594 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:26.594+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:26.594 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:26.594+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:26.594 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:26.594+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:26.863 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:26.862+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:26.863 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:26.862+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:26.863 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:26.862+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:26.864 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:26.862+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:27.071 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:27.074+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:27.072 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:27.074+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:27.072 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:27.074+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:27.072 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:27.074+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:27.370 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:27.370+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:27.371 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:27.370+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:27.371 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:27.370+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:27.371 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:27.370+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:27.371 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:27.370+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:27.570 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:27.570+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:27.570 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:27.570+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:27.570 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:27.570+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:27.570 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:27.570+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:27.882 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:27.882+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:27.882 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:27.882+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:27.882 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:27.882+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:27.882 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:27.882+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:28.028 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:28.030+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:28.029 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:28.030+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:28.029 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:28.030+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:28.029 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:28.030+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:28.375 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:01:28.378+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1745 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:01:28.386 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:28.386+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:28.386 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:28.386+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:28.386 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:28.386+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:28.387 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:28.386+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:28.387 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:28.390+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:28.542 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:28.542+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:28.542 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:28.542+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:28.542 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:28.542+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:28.542 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:28.542+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:28.858 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:28.858+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:28.858 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:28.858+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:28.858 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:28.858+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:28.859 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:28.858+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:28.981 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:28.982+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:28.981 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:28.982+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:28.982 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:28.982+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:28.982 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:28.982+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:29.343 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:29.346+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:29.344 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:29.346+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:29.344 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:29.346+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:29.344 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:29.346+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:29.344 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:29.346+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:29.494 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:29.494+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:29.495 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:29.494+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:29.495 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:29.494+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:29.495 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:29.494+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:29.874 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:29.874+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:29.875 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:29.874+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:29.875 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:29.874+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:29.875 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:29.874+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:29.956 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:29.958+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:29.956 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:29.958+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:29.956 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:29.958+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:29.957 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:29.958+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:30.336 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:30.338+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:30.337 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:30.338+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:30.337 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:30.338+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:30.337 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:30.338+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:30.337 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:30.338+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:30.468 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:30.470+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:30.469 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:30.470+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:30.469 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:30.470+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:30.469 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:30.470+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:30.879 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:30.882+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:30.880 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:30.882+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:30.880 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:30.882+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:30.880 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:30.882+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:30.923 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:30.922+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:30.923 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:30.922+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:30.923 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:30.922+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:30.923 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:30.922+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:31.316 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:31.318+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:31.316 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:31.318+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:31.316 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:31.318+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:31.317 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:31.318+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:31.317 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:31.318+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:31.433 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:31.434+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:31.434 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:31.434+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:31.434 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:31.434+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:31.434 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:31.434+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:31.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:31.878+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:31.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:31.878+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:31.877 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:31.878+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:31.877 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:31.878+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:31.915 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:31.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:31.916 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:31.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:31.916 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:31.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:31.916 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:31.918+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:32.268 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:32.270+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:32.268 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:32.270+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:32.268 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:32.270+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:32.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:32.270+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:32.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:32.270+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:32.408 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:32.410+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:32.408 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:32.410+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:32.408 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:32.410+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:32.409 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:32.410+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:32.831 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:32.830+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:32.831 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:32.830+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:32.831 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:32.830+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:32.831 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:32.830+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:32.952 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:32.954+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:32.952 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:32.954+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:32.952 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:32.954+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:32.952 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:32.954+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:33.251 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:33.254+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:33.252 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:33.254+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:33.252 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:33.254+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:33.252 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:33.254+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:33.252 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:33.254+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:33.377 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:01:33.378+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1782 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:01:33.419 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:33.422+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:33.420 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:33.422+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:33.420 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:33.422+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:33.420 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:33.422+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:33.834 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:33.834+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:33.834 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:33.834+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:33.834 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:33.834+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:33.834 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:33.834+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:33.926 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:33.926+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:33.927 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:33.926+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:33.927 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:33.926+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:33.927 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:33.926+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:34.248 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:34.250+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:34.248 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:34.250+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:34.248 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:34.250+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:34.249 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:34.250+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:34.249 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:34.250+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:34.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:34.466+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:34.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:34.466+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:34.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:34.466+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:34.465 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:34.466+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:34.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:34.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:34.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:34.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:34.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:34.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:34.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:34.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:34.966 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:34.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:34.967 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:34.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:34.967 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:34.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:34.967 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:34.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:35.240 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:35.242+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:35.241 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:35.242+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:35.241 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:35.242+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:35.241 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:35.242+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:35.241 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:35.242+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:35.436 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:35.438+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:35.436 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:35.438+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:35.436 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:35.438+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:35.436 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:35.438+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:35.905 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:35.906+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:35.905 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:35.906+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:35.905 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:35.906+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:35.905 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:35.906+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:35.928 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:35.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:35.928 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:35.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:35.929 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:35.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:35.929 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:35.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:36.196 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:36.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:36.196 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:36.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:36.196 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:36.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:36.196 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:36.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:36.197 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:36.198+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:36.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:36.426+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:36.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:36.426+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:36.424 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:36.426+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:36.424 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:36.426+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:36.875 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:36.878+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:36.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:36.878+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:36.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:36.878+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:36.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:36.878+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:36.969 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:36.970+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:36.970 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:36.970+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:36.970 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:36.970+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:36.970 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:36.970+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:37.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:37.214+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:37.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:37.214+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:37.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:37.214+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:37.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:37.214+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:37.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:37.214+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:37.402 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:37.402+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:37.402 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:37.402+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:37.403 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:37.402+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:37.403 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:37.402+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:37.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:37.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:37.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:37.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:37.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:37.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:37.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:37.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:37.991 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:37.990+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:37.991 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:37.990+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:37.991 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:37.990+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:37.992 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:37.990+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:38.260 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:38.262+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:38.260 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:38.262+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:38.261 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:38.262+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:38.261 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:38.262+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:38.261 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:38.262+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:38.378 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:01:38.378+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1817 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:01:38.428 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:38.430+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:38.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:38.430+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:38.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:38.430+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:38.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:38.430+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:38.931 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:38.930+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:38.931 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:38.930+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:38.932 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:38.930+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:38.932 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:38.930+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:38.946 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:38.946+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:38.946 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:38.946+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:38.946 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:38.946+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:38.947 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:38.946+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:39.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:39.214+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:39.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:39.214+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:39.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:39.214+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:39.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:39.214+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:39.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:39.214+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:39.448 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:39.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:39.448 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:39.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:39.448 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:39.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:39.448 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:39.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:39.940 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:39.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:39.940 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:39.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:39.940 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:39.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:39.941 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:39.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:39.963 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:39.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:39.963 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:39.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:39.964 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:39.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:39.964 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:39.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:40.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:40.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:40.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:40.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:40.173 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:40.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:40.173 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:40.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:40.173 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:40.174+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:40.410 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:40.410+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:40.410 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:40.410+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:40.410 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:40.410+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:40.411 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:40.410+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:40.950 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:40.950+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:40.950 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:40.950+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:40.950 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:40.950+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:40.950 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:40.950+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:40.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:40.986+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:40.987 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:40.986+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:40.987 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:40.986+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:40.987 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:40.986+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:41.144 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:41.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:41.144 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:41.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:41.144 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:41.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:41.145 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:41.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:41.145 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:41.146+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:41.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:41.430+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:41.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:41.430+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:41.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:41.430+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:41.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:41.430+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:41.939 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:41.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:41.939 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:41.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:41.940 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:41.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:41.940 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:41.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:41.973 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:41.974+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:41.973 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:41.974+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:41.973 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:41.974+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:41.974 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:41.974+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:42.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:42.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:42.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:42.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:42.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:42.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:42.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:42.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:42.103 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:42.102+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:42.448 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:42.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:42.449 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:42.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:42.449 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:42.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:42.449 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:42.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:42.946 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:42.946+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:42.947 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:42.946+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:42.947 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:42.946+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:42.947 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:42.946+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:42.976 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:42.978+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:42.976 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:42.978+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:42.976 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:42.978+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:42.977 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:42.978+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:43.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:43.101+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:43.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:43.101+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:43.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:43.101+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:43.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:43.101+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:43.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:43.101+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:43.379 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:01:43.377+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1854 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:01:43.481 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:43.481+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:43.482 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:43.481+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:43.482 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:43.481+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:43.482 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:43.481+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:43.942 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:43.941+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:43.942 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:43.941+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:43.943 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:43.941+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:43.943 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:43.941+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:43.985 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:43.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:43.985 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:43.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:43.985 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:43.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:43.985 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:43.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:44.070 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:44.069+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:44.071 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:44.069+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:44.071 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:44.069+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:44.072 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:44.069+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:44.072 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:44.069+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:44.469 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:44.469+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:44.469 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:44.469+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:44.469 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:44.469+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:44.470 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:44.469+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:44.907 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:44.909+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:44.908 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:44.909+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:44.908 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:44.909+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:44.908 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:44.909+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:45.009 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:45.009+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:45.009 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:45.009+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:45.009 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:45.009+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:45.009 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:45.009+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:45.051 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:45.049+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:45.051 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:45.049+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:45.051 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:45.049+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:45.052 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:45.049+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:45.052 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:45.053+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:45.457 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:45.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:45.458 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:45.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:45.458 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:45.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:45.459 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:45.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:45.926 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:45.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:45.926 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:45.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:45.926 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:45.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:45.927 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:45.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:46.003 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:46.005+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:46.003 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:46.005+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:46.004 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:46.005+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:46.004 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:46.005+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:46.055 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:46.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:46.056 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:46.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:46.056 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:46.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:46.056 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:46.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:46.056 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:46.057+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:46.437 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:46.437+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:46.437 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:46.437+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:46.437 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:46.437+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:46.438 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:46.437+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:46.897 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:46.897+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:46.897 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:46.897+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:46.897 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:46.897+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:46.897 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:46.897+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:46.966 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:46.965+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:46.966 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:46.965+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:46.966 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:46.965+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:46.967 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:46.965+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:47.031 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:47.029+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:47.031 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:47.029+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:47.031 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:47.029+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:47.031 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:47.029+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:47.031 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:47.029+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:47.410 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:47.409+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:47.411 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:47.409+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:47.411 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:47.409+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:47.411 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:47.409+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:47.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:47.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:47.904 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:47.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:47.904 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:47.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:47.904 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:47.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:47.980 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:47.981+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:47.980 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:47.981+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:47.980 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:47.981+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:47.981 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:47.981+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:48.040 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:48.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:48.041 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:48.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:48.041 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:48.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:48.041 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:48.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:48.042 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:48.041+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:48.380 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:01:48.381+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1887 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:01:48.439 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:48.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:48.440 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:48.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:48.440 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:48.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:48.440 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:48.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:48.930 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:48.929+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:48.930 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:48.929+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:48.930 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:48.929+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:48.930 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:48.929+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:48.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:48.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:48.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:48.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:48.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:48.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:48.987 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:48.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:49.041 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:49.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:49.041 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:49.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:49.041 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:49.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:49.042 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:49.041+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:49.042 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:49.041+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:49.392 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:49.393+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:49.392 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:49.393+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:49.392 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:49.393+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:49.392 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:49.393+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:49.978 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:49.977+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:49.978 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:49.977+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:49.978 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:49.977+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:49.979 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:49.977+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:49.985 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:49.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:49.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:49.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:49.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:49.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:49.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:49.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:50.057 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:50.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:50.057 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:50.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:50.058 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:50.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:50.058 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:50.057+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:50.058 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:50.057+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:50.428 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:50.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:50.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:50.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:50.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:50.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:50.429 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:50.429+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:50.948 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:50.949+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:50.948 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:50.949+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:50.948 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:50.949+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:50.948 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:50.949+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:50.964 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:50.965+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:50.965 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:50.965+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:50.965 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:50.965+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:50.965 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:50.965+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:51.098 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:51.097+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:51.098 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:51.097+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:51.098 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:51.097+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:51.099 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:51.097+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:51.099 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:51.097+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:51.402 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:51.401+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:51.402 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:51.401+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:51.402 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:51.401+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:51.403 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:51.401+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:51.937 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:51.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:51.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:51.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:51.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:51.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:51.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:51.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:51.953 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:51.953+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:51.953 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:51.953+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:51.954 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:51.953+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:51.954 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:51.953+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:52.052 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:52.053+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:52.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:52.053+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:52.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:52.053+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:52.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:52.053+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:52.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:52.053+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:52.424 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:52.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:52.425 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:52.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:52.425 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:52.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:52.425 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:52.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:52.926 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:52.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:52.926 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:52.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:52.926 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:52.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:52.927 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:52.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:52.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:52.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:52.939 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:52.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:52.939 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:52.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:52.939 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:52.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:53.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:53.017+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:53.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:53.017+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:53.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:53.017+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:53.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:53.017+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:53.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:53.017+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:53.381 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:01:53.381+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1923 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:01:53.469 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:53.469+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:53.469 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:53.469+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:53.469 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:53.469+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:53.470 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:53.469+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:53.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:53.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:53.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:53.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:53.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:53.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:53.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:53.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:53.930 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:53.929+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:53.931 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:53.929+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:53.931 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:53.929+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:53.931 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:53.929+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:54.052 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:54.053+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:54.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:54.053+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:54.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:54.053+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:54.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:54.053+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:54.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:54.053+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:54.422 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:54.421+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:54.422 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:54.421+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:54.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:54.421+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:54.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:54.421+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:54.875 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:54.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:54.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:54.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:54.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:54.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:54.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:54.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:54.936 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:54.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:54.937 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:54.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:54.937 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:54.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:54.937 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:54.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:55.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:55.017+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:55.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:55.017+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:55.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:55.017+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:55.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:55.017+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:55.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:55.017+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:55.450 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:55.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:55.450 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:55.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:55.451 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:55.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:55.451 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:55.449+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:55.880 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:55.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:55.881 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:55.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:55.881 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:55.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:55.881 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:55.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:55.939 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:55.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:55.939 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:55.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:55.939 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:55.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:55.939 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:55.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:55.997 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:55.997+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:55.997 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:55.997+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:55.997 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:55.997+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:55.997 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:55.997+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:55.998 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:55.997+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:56.473 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:56.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:56.474 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:56.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:56.474 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:56.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:56.474 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:56.473+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:56.842 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:56.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:56.842 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:56.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:56.842 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:56.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:56.843 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:56.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:56.986 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:56.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:56.987 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:56.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:56.987 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:56.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:56.987 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:56.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:56.998 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:56.997+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:56.998 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:56.997+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:56.998 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:56.997+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:56.998 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:56.997+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:56.999 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:56.997+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:57.458 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:57.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:57.459 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:57.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:57.459 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:57.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:57.459 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:57.457+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:57.844 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:57.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:57.845 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:57.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:57.845 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:57.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:57.845 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:57.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:57.951 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:57.953+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:57.952 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:57.953+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:57.952 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:57.953+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:57.952 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:57.953+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:57.955 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:57.953+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:57.955 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:57.953+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:57.955 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:57.953+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:57.956 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:57.953+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:57.956 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:57.953+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:58.382 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:01:58.381+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1957 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:01:58.491 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:58.489+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:58.491 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:58.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:58.492 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:58.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:58.492 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:58.493+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:58.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:58.873+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:58.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:58.873+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:58.874 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:58.873+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:58.874 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:58.873+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:58.911 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:58.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:58.911 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:58.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:58.911 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:58.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:58.911 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:58.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:58.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:58.909+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:58.974 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:58.973+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:58.974 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:58.973+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:58.975 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:58.973+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:58.975 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:58.973+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:59.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:59.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:59.484 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:59.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:59.485 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:59.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:59.485 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:01:59.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:01:59.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:59.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:59.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:59.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:59.865 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:59.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:59.865 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:01:59.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:01:59.907 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:59.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:59.908 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:59.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:59.908 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:59.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:59.908 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:59.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:01:59.909 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:01:59.909+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:01:59.930 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:59.933+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:59.931 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:59.933+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:59.931 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:59.933+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:01:59.931 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:01:59.933+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:00.454 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:00.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:00.454 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:00.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:00.454 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:00.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:00.455 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:00.453+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:00.843 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:00.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:00.844 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:00.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:00.844 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:00.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:00.844 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:00.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:00.896 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:00.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:00.896 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:00.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:00.896 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:00.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:00.897 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:00.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:00.897 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:00.897+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:00.953 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:00.953+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:00.953 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:00.953+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:00.953 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:00.953+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:00.954 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:00.953+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:01.432 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:01.433+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:01.432 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:01.433+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:01.433 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:01.433+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:01.433 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:01.433+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:01.856 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:01.857+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:01.856 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:01.857+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:01.856 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:01.857+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:01.857 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:01.857+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:01.857 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:01.857+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:01.880 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:01.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:01.881 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:01.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:01.881 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:01.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:01.881 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:01.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:01.957 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:01.957+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:01.958 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:01.957+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:01.958 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:01.957+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:01.958 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:01.957+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:02.396 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:02.397+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:02.396 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:02.397+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:02.396 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:02.397+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:02.396 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:02.397+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:02.845 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:02.845+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:02.845 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:02.845+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:02.845 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:02.845+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:02.846 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:02.845+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:02.846 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:02.845+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:02.899 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:02.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:02.899 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:02.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:02.900 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:02.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:02.900 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:02.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:02.920 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:02.921+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:02.920 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:02.921+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:02.921 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:02.921+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:02.921 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:02.921+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:03.383 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:02:03.385+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 1993 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:02:03.401 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:03.401+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:03.401 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:03.401+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:03.401 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:03.401+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:03.402 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:03.401+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:03.868 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:03.869+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:03.868 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:03.869+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:03.868 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:03.869+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:03.868 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:03.869+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:03.869 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:03.869+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:03.891 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:03.893+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:03.891 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:03.893+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:03.892 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:03.893+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:03.892 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:03.893+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:03.900 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:03.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:03.900 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:03.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:03.900 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:03.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:03.901 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:03.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:04.390 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:04.389+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:04.390 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:04.389+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:04.391 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:04.389+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:04.391 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:04.389+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:04.882 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:04.885+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:04.883 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:04.885+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:04.883 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:04.885+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:04.883 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:04.885+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:04.893 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:04.893+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:04.893 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:04.893+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:04.894 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:04.893+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:04.894 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:04.893+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:04.897 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:04.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:04.897 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:04.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:04.898 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:04.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:04.898 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:04.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:04.898 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:04.897+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:05.425 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:05.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:05.426 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:05.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:05.426 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:05.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:05.426 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:05.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:05.881 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:05.881+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:05.881 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:05.881+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:05.881 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:05.881+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:05.881 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:05.881+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:05.882 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:05.881+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:05.904 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:05.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:05.904 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:05.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:05.904 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:05.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:05.905 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:05.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:05.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:05.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:05.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:05.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:05.938 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:05.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:05.939 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:05.937+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:06.376 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:06.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:06.377 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:06.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:06.377 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:06.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:06.377 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:06.377+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:06.850 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:06.853+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:06.851 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:06.853+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:06.851 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:06.853+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:06.851 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:06.853+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:06.851 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:06.853+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:06.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:06.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:06.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:06.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:06.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:06.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:06.904 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:06.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:06.961 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:06.961+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:06.962 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:06.961+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:06.962 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:06.961+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:06.962 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:06.961+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:07.412 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:07.413+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:07.412 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:07.413+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:07.412 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:07.413+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:07.413 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:07.413+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:07.842 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:07.845+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:07.843 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:07.845+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:07.843 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:07.845+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:07.843 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:07.845+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:07.843 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:07.845+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:07.863 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:07.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:07.863 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:07.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:07.863 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:07.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:07.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:07.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:08.001 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:08.001+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:08.001 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:08.001+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:08.001 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:08.001+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:08.002 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:08.001+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:08.384 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:02:08.385+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2026 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:02:08.413 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:08.413+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:08.414 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:08.413+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:08.414 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:08.413+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:08.414 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:08.413+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:08.832 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:08.833+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:08.832 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:08.833+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:08.833 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:08.833+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:08.833 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:08.833+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:08.852 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:08.853+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:08.852 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:08.853+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:08.852 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:08.853+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:08.852 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:08.853+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:08.853 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:08.853+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:08.983 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:08.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:08.984 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:08.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:08.984 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:08.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:08.984 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:08.985+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:09.386 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:09.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:09.386 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:09.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:09.386 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:09.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:09.386 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:09.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:09.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:09.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:09.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:09.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:09.799 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:09.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:09.799 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:09.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:09.851 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:09.853+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:09.852 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:09.853+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:09.852 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:09.853+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:09.852 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:09.853+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:09.852 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:09.853+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:10.023 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:10.025+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:10.024 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:10.025+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:10.024 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:10.025+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:10.024 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:10.025+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:10.396 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:10.397+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:10.397 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:10.397+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:10.397 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:10.397+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:10.397 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:10.397+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:10.802 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:10.801+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:10.803 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:10.801+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:10.803 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:10.801+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:10.803 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:10.801+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:10.856 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:10.857+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:10.857 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:10.857+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:10.857 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:10.857+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:10.857 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:10.857+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:10.857 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:10.857+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:11.050 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:11.049+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:11.051 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:11.049+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:11.051 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:11.049+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:11.051 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:11.049+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:11.400 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:11.401+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:11.400 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:11.401+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:11.400 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:11.401+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:11.400 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:11.401+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:11.756 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:11.757+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:11.757 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:11.757+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:11.757 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:11.757+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:11.757 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:11.757+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:11.860 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:11.861+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:11.860 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:11.861+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:11.860 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:11.861+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:11.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:11.861+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:11.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:11.861+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:12.064 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:12.065+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:12.064 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:12.065+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:12.064 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:12.065+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:12.064 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:12.065+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:12.383 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:12.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:12.384 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:12.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:12.384 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:12.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:12.384 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:12.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:12.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:12.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:12.799 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:12.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:12.799 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:12.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:12.799 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:12.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:12.885 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:12.885+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:12.885 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:12.885+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:12.886 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:12.885+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:12.886 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:12.885+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:12.886 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:12.885+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:13.071 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:13.069+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:13.071 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:13.069+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:13.072 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:13.069+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:13.072 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:13.069+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:13.386 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:02:13.385+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2061 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:02:13.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:13.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:13.424 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:13.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:13.424 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:13.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:13.424 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:13.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:13.783 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:13.785+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:13.783 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:13.785+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:13.784 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:13.785+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:13.784 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:13.785+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:13.890 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:13.889+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:13.890 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:13.889+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:13.891 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:13.889+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:13.891 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:13.889+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:13.891 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:13.889+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:14.089 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:14.089+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:14.090 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:14.089+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:14.090 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:14.089+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:14.090 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:14.089+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:14.392 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:14.393+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:14.393 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:14.393+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:14.393 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:14.393+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:14.393 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:14.393+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:14.745 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:14.745+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:14.745 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:14.745+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:14.746 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:14.745+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:14.746 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:14.745+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:14.914 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:14.917+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:14.914 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:14.917+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:14.915 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:14.917+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:14.915 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:14.917+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:14.915 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:14.917+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:15.130 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:15.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:15.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:15.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:15.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:15.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:15.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:15.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:15.362 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:15.361+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:15.362 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:15.361+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:15.362 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:15.361+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:15.363 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:15.361+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:15.706 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:15.705+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:15.706 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:15.705+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:15.707 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:15.705+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:15.707 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:15.705+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:15.904 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:15.905+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:15.904 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:15.905+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:15.904 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:15.905+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:15.905 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:15.905+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:15.905 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:15.905+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:16.110 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:16.109+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:16.111 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:16.109+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:16.111 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:16.109+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:16.111 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:16.109+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:16.334 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:16.333+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:16.334 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:16.333+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:16.334 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:16.333+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:16.334 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:16.333+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:16.714 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:16.713+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:16.714 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:16.713+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:16.715 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:16.713+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:16.715 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:16.713+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:16.906 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:16.905+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:16.906 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:16.905+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:16.906 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:16.905+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:16.906 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:16.905+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:16.907 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:16.905+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:17.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:17.125+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:17.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:17.125+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:17.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:17.125+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:17.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:17.125+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:17.293 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:17.293+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:17.293 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:17.293+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:17.293 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:17.293+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:17.293 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:17.293+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:17.709 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:17.709+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:17.710 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:17.709+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:17.710 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:17.709+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:17.710 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:17.709+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:17.937 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:17.937+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:17.937 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:17.937+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:17.937 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:17.937+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:17.937 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:17.937+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:17.938 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:17.937+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:18.141 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:18.141+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:18.143 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:18.141+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:18.143 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:18.141+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:18.143 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:18.141+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:18.314 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:18.313+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:18.314 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:18.313+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:18.314 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:18.313+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:18.315 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:18.313+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:18.386 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:02:18.385+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2096 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:02:18.673 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:18.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:18.674 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:18.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:18.674 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:18.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:18.674 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:18.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:18.893 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:18.893+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:18.893 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:18.893+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:18.893 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:18.893+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:18.893 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:18.893+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:18.894 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:18.893+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:19.167 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:19.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:19.168 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:19.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:19.168 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:19.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:19.168 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:19.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:19.283 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:19.281+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:19.283 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:19.281+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:19.283 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:19.281+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:19.284 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:19.281+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:19.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:19.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:19.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:19.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:19.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:19.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:19.672 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:19.673+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:19.903 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:19.905+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:19.904 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:19.905+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:19.904 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:19.905+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:19.904 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:19.905+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:19.904 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:19.905+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:20.200 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:20.201+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:20.201 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:20.201+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:20.201 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:20.201+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:20.201 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:20.201+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:20.317 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:20.317+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:20.317 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:20.317+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:20.317 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:20.317+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:20.318 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:20.317+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:20.628 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:20.629+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:20.629 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:20.629+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:20.629 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:20.629+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:20.629 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:20.629+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:20.930 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:20.929+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:20.930 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:20.929+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:20.930 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:20.929+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:20.931 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:20.929+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:20.931 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:20.929+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:21.182 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:21.185+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:21.183 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:21.185+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:21.183 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:21.185+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:21.183 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:21.185+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:21.272 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:21.273+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:21.272 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:21.273+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:21.272 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:21.273+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:21.272 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:21.273+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:21.609 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:21.609+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:21.609 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:21.609+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:21.610 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:21.609+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:21.610 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:21.609+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:21.943 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:21.941+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:21.943 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:21.941+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:21.943 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:21.941+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:21.944 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:21.941+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:21.944 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:21.941+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:22.222 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:22.220+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:22.222 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:22.220+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:22.223 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:22.220+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:22.223 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:22.220+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:22.230 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:22.228+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:22.230 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:22.228+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:22.231 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:22.228+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:22.231 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:22.228+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:22.610 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:22.608+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:22.610 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:22.608+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:22.610 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:22.608+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:22.611 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:22.608+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:22.938 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:22.936+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:22.938 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:22.936+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:22.939 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:22.936+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:22.939 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:22.936+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:22.939 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:22.936+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:23.243 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:23.244+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:23.244 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:23.244+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:23.244 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:23.244+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:23.244 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:23.244+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:23.248 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:23.248+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:23.249 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:23.248+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:23.249 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:23.248+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:23.249 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:23.248+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:23.387 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:02:23.384+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2131 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:02:23.621 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:23.620+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:23.622 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:23.620+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:23.622 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:23.620+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:23.622 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:23.620+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:23.952 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:23.952+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:23.952 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:23.952+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:23.952 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:23.952+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:23.953 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:23.952+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:23.953 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:23.952+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:24.259 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:24.260+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:24.260 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:24.260+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:24.260 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:24.260+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:24.260 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:24.260+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:24.268 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:24.268+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:24.268 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:24.268+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:24.268 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:24.268+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:24.269 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:24.268+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:24.635 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:24.632+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:24.636 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:24.632+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:24.636 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:24.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:24.636 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:24.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:24.966 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:24.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:24.967 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:24.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:24.967 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:24.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:24.967 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:24.968+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:24.968 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:24.968+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:25.224 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:25.224+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:25.224 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:25.224+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:25.225 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:25.224+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:25.225 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:25.224+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:25.302 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:25.300+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:25.302 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:25.300+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:25.302 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:25.300+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:25.302 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:25.300+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:25.638 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:25.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:25.639 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:25.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:25.639 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:25.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:25.639 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:25.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:25.961 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:25.960+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:25.962 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:25.960+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:25.962 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:25.960+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:25.962 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:25.960+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:25.962 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:25.960+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:26.208 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:26.208+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:26.208 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:26.208+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:26.209 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:26.208+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:26.209 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:26.208+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:26.342 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:26.340+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:26.342 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:26.340+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:26.343 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:26.340+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:26.343 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:26.340+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:26.685 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:26.684+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:26.686 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:26.684+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:26.686 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:26.684+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:26.686 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:26.684+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:26.957 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:26.956+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:26.957 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:26.956+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:26.957 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:26.956+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:26.958 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:26.956+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:26.958 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:26.956+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:27.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:27.248+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:27.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:27.248+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:27.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:27.248+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:27.249 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:27.248+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:27.342 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:27.340+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:27.342 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:27.340+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:27.342 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:27.340+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:27.342 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:27.340+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:27.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:27.672+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:27.672 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:27.672+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:27.672 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:27.672+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:27.672 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:27.672+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:27.925 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:27.924+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:27.925 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:27.924+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:27.925 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:27.924+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:27.926 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:27.924+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:27.926 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:27.924+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:28.258 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:28.256+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:28.259 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:28.256+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:28.259 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:28.256+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:28.259 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:28.256+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:28.333 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:28.332+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:28.333 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:28.332+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:28.334 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:28.332+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:28.334 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:28.332+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:28.389 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:02:28.388+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2165 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:02:28.693 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:28.692+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:28.694 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:28.692+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:28.694 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:28.692+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:28.694 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:28.692+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:28.946 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:28.944+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:28.947 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:28.944+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:28.947 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:28.944+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:28.947 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:28.944+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:28.947 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:28.944+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:29.239 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:29.240+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:29.240 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:29.240+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:29.240 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:29.240+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:29.240 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:29.240+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:29.321 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:29.320+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:29.322 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:29.320+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:29.322 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:29.320+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:29.322 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:29.320+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:29.729 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:29.728+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:29.730 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:29.728+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:29.730 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:29.728+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:29.731 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:29.728+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:29.979 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:29.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:29.979 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:29.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:29.979 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:29.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:29.980 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:29.980+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:29.980 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:29.980+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:30.286 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:30.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:30.286 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:30.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:30.287 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:30.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:30.287 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:30.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:30.365 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:30.364+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:30.366 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:30.364+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:30.366 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:30.364+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:30.366 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:30.364+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:30.715 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:30.716+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:30.715 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:30.716+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:30.716 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:30.716+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:30.716 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:30.716+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:31.007 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:31.008+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:31.007 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:31.008+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:31.007 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:31.008+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:31.007 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:31.008+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:31.008 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:31.008+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:31.276 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:31.276+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:31.276 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:31.276+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:31.277 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:31.276+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:31.277 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:31.276+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:31.404 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:31.404+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:31.405 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:31.404+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:31.405 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:31.404+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:31.405 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:31.404+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:31.668 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:31.668+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:31.668 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:31.668+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:31.669 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:31.668+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:31.669 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:31.668+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:32.049 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:32.048+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:32.049 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:32.048+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:32.049 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:32.048+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:32.050 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:32.048+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:32.050 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:32.048+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:32.315 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:32.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:32.315 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:32.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:32.315 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:32.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:32.315 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:32.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:32.431 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:32.432+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:32.432 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:32.432+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:32.432 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:32.432+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:32.432 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:32.432+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:32.641 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:32.640+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:32.641 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:32.640+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:32.641 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:32.640+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:32.642 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:32.640+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:33.072 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:33.072+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:33.073 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:33.072+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:33.073 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:33.072+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:33.073 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:33.072+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:33.073 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:33.072+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:33.288 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:33.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:33.289 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:33.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:33.289 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:33.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:33.289 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:33.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:33.389 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:02:33.388+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2200 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:02:33.395 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:33.392+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:33.396 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:33.392+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:33.396 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:33.392+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:33.397 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:33.392+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:33.648 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:33.648+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:33.648 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:33.648+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:33.648 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:33.648+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:33.648 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:33.648+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:34.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:34.104+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:34.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:34.104+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:34.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:34.104+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:34.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:34.104+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:34.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:34.104+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:34.290 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:34.292+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:34.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:34.292+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:34.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:34.292+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:34.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:34.292+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:34.396 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:34.396+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:34.396 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:34.396+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:34.397 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:34.396+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:34.397 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:34.396+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:34.661 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:34.660+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:34.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:34.660+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:34.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:34.660+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:34.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:34.660+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:35.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:35.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:35.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:35.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:35.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:35.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:35.126 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:35.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:35.126 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:35.124+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:35.328 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:35.328+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:35.328 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:35.328+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:35.329 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:35.328+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:35.329 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:35.328+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:35.425 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:35.424+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:35.425 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:35.424+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:35.425 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:35.424+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:35.426 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:35.424+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:35.707 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:35.708+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:35.707 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:35.708+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:35.707 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:35.708+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:35.708 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:35.708+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:36.134 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:36.132+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:36.134 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:36.132+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:36.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:36.132+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:36.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:36.132+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:36.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:36.132+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:36.306 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:36.308+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:36.306 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:36.308+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:36.307 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:36.308+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:36.307 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:36.308+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:36.389 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:36.388+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:36.389 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:36.388+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:36.389 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:36.388+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:36.389 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:36.388+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:36.665 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:36.664+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:36.666 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:36.664+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:36.666 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:36.664+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:36.666 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:36.664+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:37.169 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:37.168+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:37.169 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:37.168+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:37.170 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:37.168+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:37.170 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:37.168+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:37.170 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:37.168+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:37.309 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:37.308+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:37.309 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:37.308+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:37.309 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:37.308+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:37.310 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:37.308+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:37.428 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:37.428+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:37.428 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:37.428+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:37.428 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:37.428+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:37.428 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:37.428+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:37.617 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:37.616+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:37.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:37.616+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:37.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:37.616+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:37.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:37.616+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:38.154 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:38.152+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:38.154 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:38.152+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:38.154 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:38.152+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:38.155 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:38.152+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:38.155 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:38.152+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:38.283 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:38.280+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:38.284 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:38.280+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:38.284 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:38.280+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:38.284 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:38.280+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:38.391 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:02:38.388+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2235 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:02:38.401 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:38.400+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:38.401 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:38.400+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:38.401 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:38.400+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:38.401 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:38.400+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:38.604 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:38.604+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:38.604 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:38.604+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:38.604 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:38.604+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:38.604 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:38.604+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:39.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:39.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:39.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:39.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:39.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:39.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:39.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:39.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:39.126 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:39.124+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:39.253 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:39.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:39.253 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:39.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:39.254 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:39.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:39.254 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:39.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:39.411 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:39.412+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:39.412 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:39.412+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:39.412 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:39.412+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:39.412 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:39.412+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:39.561 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:39.560+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:39.562 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:39.560+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:39.562 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:39.560+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:39.562 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:39.560+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:40.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:40.104+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:40.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:40.104+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:40.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:40.104+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:40.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:40.104+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:40.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:40.104+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:40.268 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:40.268+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:40.269 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:40.268+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:40.269 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:40.268+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:40.269 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:40.268+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:40.411 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:40.412+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:40.412 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:40.412+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:40.412 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:40.412+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:40.412 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:40.412+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:40.559 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:40.560+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:40.559 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:40.560+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:40.559 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:40.560+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:40.560 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:40.560+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:41.077 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:41.076+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:41.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:41.076+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:41.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:41.076+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:41.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:41.076+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:41.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:41.076+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:41.313 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:41.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:41.313 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:41.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:41.313 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:41.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:41.314 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:41.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:41.369 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:41.368+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:41.370 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:41.368+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:41.370 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:41.368+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:41.370 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:41.368+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:41.541 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:41.540+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:41.541 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:41.540+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:41.541 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:41.540+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:41.541 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:41.540+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:42.033 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:42.032+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:42.079 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:42.032+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:42.080 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:42.032+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:42.080 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:42.032+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:42.080 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:42.032+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:42.289 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:42.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:42.289 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:42.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:42.289 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:42.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:42.290 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:42.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:42.321 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:42.320+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:42.321 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:42.320+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:42.321 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:42.320+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:42.321 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:42.320+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:42.504 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:42.504+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:42.504 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:42.504+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:42.505 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:42.504+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:42.505 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:42.504+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:43.023 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:43.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:43.024 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:43.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:43.024 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:43.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:43.024 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:43.024+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:43.024 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:43.024+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:43.312 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:43.312+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:43.313 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:43.312+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:43.313 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:43.312+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:43.313 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:43.312+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:43.323 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:43.320+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:43.323 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:43.320+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:43.323 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:43.320+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:43.323 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:43.320+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:43.392 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:02:43.392+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2270 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:02:43.509 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:43.508+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:43.509 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:43.508+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:43.509 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:43.508+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:43.510 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:43.508+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:44.061 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:44.060+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:44.061 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:44.060+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:44.061 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:44.060+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:44.062 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:44.060+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:44.062 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:44.060+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:44.313 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:44.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:44.313 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:44.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:44.314 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:44.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:44.314 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:44.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:44.331 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:44.332+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:44.332 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:44.332+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:44.332 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:44.332+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:44.332 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:44.332+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:44.473 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:44.472+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:44.474 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:44.472+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:44.474 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:44.472+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:44.474 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:44.472+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:45.050 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:45.048+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:45.050 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:45.048+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:45.050 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:45.048+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:45.051 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:45.048+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:45.051 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:45.048+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:45.312 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:45.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:45.312 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:45.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:45.313 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:45.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:45.313 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:45.312+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:45.342 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:45.340+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:45.343 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:45.340+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:45.343 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:45.340+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:45.343 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:45.340+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:45.443 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:45.444+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:45.444 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:45.444+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:45.444 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:45.444+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:45.444 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:45.444+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:46.083 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:46.084+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:46.084 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:46.084+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:46.084 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:46.084+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:46.084 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:46.084+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:46.084 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:46.084+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:46.302 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:46.304+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:46.302 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:46.304+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:46.303 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:46.304+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:46.303 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:46.304+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:46.388 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:46.388+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:46.389 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:46.388+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:46.389 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:46.388+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:46.389 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:46.388+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:46.452 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:46.452+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:46.452 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:46.452+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:46.452 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:46.452+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:46.452 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:46.452+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:47.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:47.052+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:47.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:47.052+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:47.054 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:47.052+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:47.054 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:47.052+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:47.054 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:47.052+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:47.306 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:47.308+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:47.306 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:47.308+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:47.306 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:47.308+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:47.306 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:47.308+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:47.358 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:47.360+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:47.359 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:47.360+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:47.359 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:47.360+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:47.359 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:47.360+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:47.477 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:47.476+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:47.477 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:47.476+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:47.478 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:47.476+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:47.478 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:47.476+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:48.100 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:48.100+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:48.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:48.100+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:48.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:48.100+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:48.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:48.100+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:48.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:48.100+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:48.331 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:48.332+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:48.331 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:48.332+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:48.331 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:48.332+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:48.332 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:48.332+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:48.333 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:48.332+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:48.333 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:48.332+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:48.333 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:48.332+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:48.334 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:48.332+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:48.393 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:02:48.392+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2304 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:02:48.497 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:48.496+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:48.497 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:48.496+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:48.498 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:48.496+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:48.498 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:48.496+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:49.111 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:49.108+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:49.111 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:49.108+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:49.112 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:49.108+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:49.112 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:49.108+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:49.112 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:49.112+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:49.292 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:49.292+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:49.293 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:49.292+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:49.293 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:49.292+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:49.293 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:49.292+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:49.346 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:49.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:49.347 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:49.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:49.347 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:49.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:49.347 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:49.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:49.459 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:49.460+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:49.459 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:49.460+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:49.460 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:49.460+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:49.460 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:49.460+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:50.064 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:50.064+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:50.065 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:50.064+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:50.065 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:50.064+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:50.065 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:50.064+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:50.065 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:50.064+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:50.324 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:50.324+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:50.325 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:50.324+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:50.325 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:50.324+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:50.325 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:50.324+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:50.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:50.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:50.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:50.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:50.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:50.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:50.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:50.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:50.423 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:50.424+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:50.423 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:50.424+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:50.424 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:50.424+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:50.424 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:50.424+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:51.077 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:51.076+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:51.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:51.076+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:51.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:51.076+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:51.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:51.076+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:51.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:51.076+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:51.344 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:51.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:51.345 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:51.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:51.345 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:51.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:51.345 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:51.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:51.359 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:51.356+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:51.359 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:51.356+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:51.359 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:51.356+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:51.360 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:51.356+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:51.442 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:51.444+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:51.442 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:51.444+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:51.442 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:51.444+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:51.443 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:51.444+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:52.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:52.108+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:52.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:52.108+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:52.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:52.108+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:52.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:52.108+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:52.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:52.108+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:52.327 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:52.328+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:52.327 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:52.328+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:52.327 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:52.328+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:52.327 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:52.328+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:52.339 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:52.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:52.339 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:52.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:52.340 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:52.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:52.340 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:52.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:52.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:52.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:52.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:52.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:52.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:52.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:52.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:52.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:53.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:53.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:53.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:53.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:53.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:53.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:53.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:53.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:53.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:53.124+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:53.331 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:53.328+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:53.331 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:53.328+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:53.331 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:53.328+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:53.332 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:53.328+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:53.378 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:53.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:53.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:53.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:53.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:53.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:53.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:53.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:53.394 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:02:53.392+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2339 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:02:53.407 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:53.408+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:53.407 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:53.408+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:53.408 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:53.408+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:53.408 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:53.408+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:54.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:54.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:54.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:54.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:54.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:54.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:54.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:54.124+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:54.123 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:54.124+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:54.302 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:54.304+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:54.302 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:54.304+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:54.303 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:54.304+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:54.303 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:54.304+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:54.340 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:54.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:54.340 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:54.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:54.340 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:54.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:54.341 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:54.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:54.414 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:54.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:54.415 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:54.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:54.415 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:54.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:54.415 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:54.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:55.149 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:55.144+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:55.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:55.144+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:55.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:55.144+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:55.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:55.144+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:55.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:55.144+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:55.297 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:55.296+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:55.298 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:55.296+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:55.298 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:55.296+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:55.298 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:55.296+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:55.299 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:55.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:55.299 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:55.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:55.299 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:55.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:55.299 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:55.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:55.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:55.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:55.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:55.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:55.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:55.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:55.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:55.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:56.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:56.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:56.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:56.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:56.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:56.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:56.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:56.148+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:56.149 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:56.148+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:56.311 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:56.312+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:56.311 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:56.312+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:56.311 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:56.312+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:56.312 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:56.312+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:56.341 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:56.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:56.342 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:56.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:56.342 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:56.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:56.342 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:56.344+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:56.434 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:56.436+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:56.434 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:56.436+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:56.434 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:56.436+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:56.435 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:56.436+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:57.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:57.108+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:57.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:57.108+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:57.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:57.108+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:57.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:57.108+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:57.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:57.108+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:57.282 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:57.280+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:57.283 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:57.280+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:57.283 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:57.280+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:57.283 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:57.280+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:57.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:57.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:57.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:57.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:57.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:57.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:57.373 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:57.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:57.465 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:57.464+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:57.465 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:57.464+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:57.465 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:57.464+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:57.466 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:57.464+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:58.072 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:58.072+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:58.073 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:58.072+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:58.073 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:58.072+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:58.073 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:58.072+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:58.073 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:58.072+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:58.258 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:58.256+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:58.259 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:58.256+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:58.259 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:58.256+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:58.259 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:58.256+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:58.374 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:58.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:58.374 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:58.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:58.375 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:58.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:58.375 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:58.372+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:58.395 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:02:58.396+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2375 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:02:58.460 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:58.460+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:58.460 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:58.460+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:58.460 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:58.460+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:58.460 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:58.460+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:59.058 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:59.060+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:59.059 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:59.060+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:59.059 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:59.060+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:59.059 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:59.060+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:02:59.059 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:02:59.060+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:02:59.287 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:59.284+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:59.287 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:59.284+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:59.287 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:59.284+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:59.288 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:02:59.284+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:02:59.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:59.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:59.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:59.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:59.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:59.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:59.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:02:59.376+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:02:59.450 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:59.452+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:59.450 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:59.452+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:59.450 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:59.452+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:02:59.451 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:02:59.452+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:00.069 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:00.068+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:00.069 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:00.068+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:00.069 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:00.068+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:00.069 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:00.068+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:00.070 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:00.068+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:00.262 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:00.260+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:00.262 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:00.260+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:00.262 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:00.260+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:00.263 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:00.260+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:00.418 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:00.420+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:00.418 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:00.420+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:00.418 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:00.420+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:00.418 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:00.420+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:00.441 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:00.440+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:00.441 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:00.440+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:00.441 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:00.440+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:00.441 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:00.440+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:01.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:01.044+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:01.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:01.044+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:01.044 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:01.044+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:01.044 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:01.044+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:01.044 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:01.044+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:01.274 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:01.276+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:01.274 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:01.276+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:01.275 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:01.276+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:01.275 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:01.276+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:01.377 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:01.380+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:01.378 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:01.380+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:01.378 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:01.380+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:01.378 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:01.380+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:01.447 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:01.448+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:01.447 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:01.448+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:01.447 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:01.448+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:01.448 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:01.448+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:02.010 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:02.008+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:02.010 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:02.008+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:02.010 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:02.008+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:02.010 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:02.008+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:02.011 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:02.008+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:02.284 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:02.284+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:02.284 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:02.284+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:02.285 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:02.284+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:02.285 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:02.284+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:02.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:02.336+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:02.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:02.336+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:02.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:02.336+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:02.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:02.336+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:02.424 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:02.424+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:02.425 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:02.424+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:02.425 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:02.424+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:02.425 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:02.424+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:03.016 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:03.016+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:03.016 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:03.016+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:03.016 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:03.016+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:03.017 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:03.016+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:03.017 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:03.016+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:03.264 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:03.264+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:03.265 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:03.264+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:03.265 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:03.264+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:03.265 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:03.264+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:03.304 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:03.304+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:03.304 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:03.304+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:03.305 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:03.304+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:03.305 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:03.304+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:03.397 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:03:03.396+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2411 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:03:03.461 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:03.460+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:03.461 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:03.460+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:03.462 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:03.460+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:03.462 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:03.460+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:03.977 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:03.976+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:03.977 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:03.976+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:03.977 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:03.976+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:03.978 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:03.976+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:03.978 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:03.976+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:04.274 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:04.275+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:04.274 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:04.275+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:04.275 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:04.275+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:04.275 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:04.275+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:04.308 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:04.307+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:04.308 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:04.307+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:04.308 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:04.307+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:04.308 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:04.307+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:04.472 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:04.471+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:04.472 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:04.471+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:04.473 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:04.471+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:04.473 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:04.471+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:05.011 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:05.011+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:05.011 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:05.011+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:05.011 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:05.011+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:05.012 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:05.011+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:05.012 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:05.011+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:05.263 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:05.263+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:05.264 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:05.263+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:05.264 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:05.263+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:05.264 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:05.263+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:05.343 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:05.343+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:05.344 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:05.343+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:05.344 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:05.343+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:05.344 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:05.343+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:05.459 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:05.459+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:05.459 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:05.459+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:05.460 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:05.459+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:05.460 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:05.459+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:06.001 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:05.999+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:06.002 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:05.999+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:06.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:05.999+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:06.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:05.999+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:06.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:05.999+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:06.255 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:06.251+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:06.255 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:06.251+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:06.256 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:06.251+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:06.256 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:06.251+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:06.343 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:06.343+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:06.344 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:06.343+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:06.344 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:06.343+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:06.344 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:06.343+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:06.485 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:06.483+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:06.485 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:06.483+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:06.486 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:06.483+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:06.486 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:06.483+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:06.977 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:06.979+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:06.978 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:06.979+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:06.978 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:06.979+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:06.978 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:06.979+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:06.978 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:06.979+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:07.290 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:07.287+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:07.290 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:07.287+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:07.290 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:07.287+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:07.291 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:07.287+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:07.383 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:07.383+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:07.384 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:07.383+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:07.384 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:07.383+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:07.384 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:07.383+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:07.459 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:07.455+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:07.459 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:07.455+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:07.459 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:07.455+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:07.460 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:07.455+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:08.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:08.023+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:08.027 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:08.023+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:08.027 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:08.023+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:08.027 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:08.023+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:08.028 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:08.023+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:08.242 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:08.243+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:08.242 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:08.243+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:08.242 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:08.243+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:08.243 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:08.243+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:08.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:08.335+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:08.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:08.335+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:08.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:08.335+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:08.337 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:08.335+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:08.398 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:03:08.395+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2446 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:03:08.478 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:08.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:08.478 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:08.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:08.479 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:08.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:08.479 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:08.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:09.048 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:09.047+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:09.048 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:09.047+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:09.048 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:09.047+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:09.049 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:09.047+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:09.049 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:09.047+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:09.258 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:09.255+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:09.258 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:09.255+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:09.258 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:09.255+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:09.258 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:09.255+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:09.327 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:09.327+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:09.328 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:09.327+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:09.328 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:09.327+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:09.328 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:09.327+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:09.455 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:09.455+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:09.455 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:09.455+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:09.455 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:09.455+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:09.455 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:09.455+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:10.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:09.999+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:10.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:09.999+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:10.004 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:09.999+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:10.004 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:09.999+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:10.004 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:09.999+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:10.242 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:10.243+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:10.243 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:10.243+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:10.243 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:10.243+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:10.244 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:10.243+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:10.295 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:10.291+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:10.295 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:10.291+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:10.295 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:10.291+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:10.295 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:10.291+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:10.491 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:10.491+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:10.492 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:10.491+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:10.492 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:10.491+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:10.492 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:10.491+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:11.040 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:11.039+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:11.041 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:11.039+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:11.041 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:11.039+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:11.041 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:11.039+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:11.041 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:11.039+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:11.222 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:11.219+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:11.223 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:11.219+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:11.223 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:11.219+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:11.223 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:11.219+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:11.293 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:11.291+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:11.294 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:11.291+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:11.294 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:11.291+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:11.294 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:11.291+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:11.490 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:11.487+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:11.490 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:11.487+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:11.491 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:11.487+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:11.491 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:11.487+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:12.006 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:12.007+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:12.006 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:12.007+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:12.006 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:12.007+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:12.007 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:12.007+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:12.007 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:12.007+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:12.224 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:12.223+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:12.224 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:12.223+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:12.225 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:12.223+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:12.225 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:12.223+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:12.249 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:12.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:12.249 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:12.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:12.249 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:12.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:12.249 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:12.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:12.462 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:12.459+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:12.462 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:12.459+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:12.462 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:12.459+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:12.463 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:12.459+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:13.035 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:13.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:13.035 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:13.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:13.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:13.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:13.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:13.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:13.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:13.035+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:13.230 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:13.227+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:13.231 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:13.227+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:13.231 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:13.227+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:13.231 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:13.227+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:13.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:13.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:13.251 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:13.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:13.251 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:13.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:13.251 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:13.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:13.400 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:03:13.399+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2482 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:03:13.510 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:13.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:13.510 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:13.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:13.511 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:13.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:13.511 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:13.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:14.054 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:14.051+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:14.054 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:14.051+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:14.055 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:14.051+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:14.055 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:14.051+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:14.055 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:14.051+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:14.256 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:14.255+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:14.257 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:14.255+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:14.257 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:14.255+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:14.257 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:14.255+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:14.258 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:14.255+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:14.258 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:14.255+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:14.258 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:14.255+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:14.259 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:14.255+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:14.553 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:14.551+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:14.553 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:14.551+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:14.553 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:14.551+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:14.554 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:14.551+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:15.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:15.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:15.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:15.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:15.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:15.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:15.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:15.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:15.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:15.019+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:15.223 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:15.223+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:15.224 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:15.223+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:15.224 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:15.223+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:15.224 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:15.223+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:15.300 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:15.299+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:15.300 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:15.299+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:15.300 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:15.299+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:15.300 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:15.299+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:15.594 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:15.595+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:15.594 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:15.595+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:15.594 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:15.595+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:15.595 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:15.595+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:16.035 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:16.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:16.035 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:16.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:16.035 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:16.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:16.035 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:16.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:16.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:16.031+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:16.230 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:16.227+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:16.231 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:16.227+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:16.231 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:16.227+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:16.231 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:16.227+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:16.258 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:16.255+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:16.258 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:16.255+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:16.259 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:16.255+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:16.259 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:16.255+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:16.588 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:16.587+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:16.588 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:16.587+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:16.588 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:16.587+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:16.589 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:16.587+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:17.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:17.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:17.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:17.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:17.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:17.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:17.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:17.035+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:17.037 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:17.035+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:17.228 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:17.227+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:17.229 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:17.227+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:17.229 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:17.227+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:17.229 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:17.227+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:17.286 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:17.283+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:17.286 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:17.283+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:17.286 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:17.283+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:17.286 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:17.283+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:17.611 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:17.611+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:17.611 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:17.611+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:17.611 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:17.611+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:17.612 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:17.611+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:17.997 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:17.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:17.997 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:17.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:17.997 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:17.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:17.998 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:17.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:17.998 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:17.995+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:18.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:18.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:18.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:18.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:18.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:18.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:18.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:18.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:18.251 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:18.251+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:18.252 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:18.251+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:18.252 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:18.251+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:18.252 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:18.251+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:18.400 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:03:18.399+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2518 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:03:18.632 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:18.631+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:18.632 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:18.631+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:18.632 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:18.631+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:18.632 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:18.631+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:18.968 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:18.967+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:18.969 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:18.967+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:18.969 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:18.967+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:18.969 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:18.967+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:18.969 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:18.967+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:19.249 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:19.251+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:19.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:19.251+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:19.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:19.251+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:19.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:19.251+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:19.275 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:19.275+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:19.276 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:19.275+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:19.276 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:19.275+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:19.276 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:19.275+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:19.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:19.671+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:19.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:19.671+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:19.672 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:19.671+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:19.672 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:19.671+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:20.015 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:20.011+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:20.015 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:20.011+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:20.015 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:20.011+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:20.016 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:20.011+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:20.016 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:20.011+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:20.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:20.243+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:20.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:20.243+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:20.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:20.243+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:20.246 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:20.243+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:20.273 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:20.271+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:20.273 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:20.271+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:20.274 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:20.271+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:20.274 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:20.271+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:20.661 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:20.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:20.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:20.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:20.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:20.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:20.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:20.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:20.989 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:20.987+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:20.989 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:20.987+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:20.989 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:20.987+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:20.990 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:20.987+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:20.990 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:20.987+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:21.228 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:21.227+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:21.228 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:21.227+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:21.228 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:21.227+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:21.228 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:21.227+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:21.247 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:21.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:21.247 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:21.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:21.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:21.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:21.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:21.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:21.625 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:21.627+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:21.626 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:21.627+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:21.626 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:21.627+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:21.626 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:21.627+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:21.958 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:21.959+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:21.959 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:21.959+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:21.959 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:21.959+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:21.959 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:21.959+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:21.959 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:21.959+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:22.203 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:22.203+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:22.204 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:22.203+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:22.204 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:22.203+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:22.204 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:22.203+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:22.272 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:22.271+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:22.272 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:22.271+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:22.273 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:22.271+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:22.273 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:22.271+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:22.659 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:22.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:22.659 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:22.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:22.659 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:22.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:22.659 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:22.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:22.924 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:22.923+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:22.924 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:22.923+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:22.925 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:22.923+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:22.925 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:22.923+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:22.925 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:22.923+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:23.155 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:23.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:23.155 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:23.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:23.155 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:23.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:23.155 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:23.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:23.260 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:23.259+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:23.260 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:23.259+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:23.260 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:23.259+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:23.261 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:23.259+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:23.401 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:03:23.399+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2554 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:03:23.611 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:23.607+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:23.611 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:23.607+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:23.611 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:23.607+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:23.611 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:23.607+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:23.955 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:23.955+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:23.955 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:23.955+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:23.955 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:23.955+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:23.955 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:23.955+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:23.956 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:23.955+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:24.119 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:24.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:24.120 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:24.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:24.120 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:24.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:24.120 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:24.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:24.287 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:24.283+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:24.287 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:24.283+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:24.288 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:24.283+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:24.288 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:24.283+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:24.614 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:24.611+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:24.615 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:24.611+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:24.615 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:24.611+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:24.615 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:24.611+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:24.990 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:24.991+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:24.991 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:24.991+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:24.991 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:24.991+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:24.991 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:24.991+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:24.991 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:24.991+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:25.095 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:25.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:25.096 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:25.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:25.096 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:25.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:25.096 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:25.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:25.332 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:25.331+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:25.333 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:25.331+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:25.333 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:25.331+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:25.333 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:25.331+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:25.614 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:25.615+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:25.615 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:25.615+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:25.615 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:25.615+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:25.615 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:25.615+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:26.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:26.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:26.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:26.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:26.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:26.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:26.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:26.019+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:26.022 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:26.019+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:26.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:26.071+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:26.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:26.071+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:26.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:26.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:26.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:26.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:26.333 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:26.335+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:26.333 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:26.335+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:26.334 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:26.335+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:26.334 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:26.335+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:26.654 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:26.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:26.655 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:26.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:26.655 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:26.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:26.655 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:26.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:26.994 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:26.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:26.994 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:26.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:26.995 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:26.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:26.995 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:26.995+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:26.995 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:26.995+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:27.026 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:27.027+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:27.026 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:27.027+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:27.027 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:27.027+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:27.027 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:27.027+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:27.335 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:27.335+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:27.336 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:27.335+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:27.336 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:27.335+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:27.336 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:27.335+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:27.651 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:27.651+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:27.652 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:27.651+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:27.652 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:27.651+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:27.652 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:27.651+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:28.032 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:28.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:28.033 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:28.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:28.033 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:28.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:28.033 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:28.031+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:28.033 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:28.031+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:28.034 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:28.031+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:28.034 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:28.031+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:28.034 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:28.031+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:28.034 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:28.031+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:28.339 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:28.335+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:28.339 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:28.335+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:28.339 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:28.335+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:28.340 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:28.335+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:28.402 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:03:28.403+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2588 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:03:28.655 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:28.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:28.656 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:28.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:28.656 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:28.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:28.656 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:28.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:28.991 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:28.991+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:28.991 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:28.991+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:28.991 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:28.991+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:28.991 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:28.991+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:28.992 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:28.991+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:29.057 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:29.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:29.058 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:29.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:29.058 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:29.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:29.058 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:29.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:29.295 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:29.295+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:29.295 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:29.295+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:29.296 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:29.295+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:29.296 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:29.295+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:29.665 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:29.663+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:29.666 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:29.663+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:29.666 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:29.663+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:29.666 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:29.663+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:29.946 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:29.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:29.947 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:29.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:29.947 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:29.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:29.947 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:29.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:29.947 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:29.947+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:30.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:30.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:30.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:30.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:30.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:30.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:30.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:30.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:30.257 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:30.259+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:30.257 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:30.259+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:30.258 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:30.259+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:30.258 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:30.259+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:30.638 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:30.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:30.639 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:30.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:30.639 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:30.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:30.639 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:30.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:30.947 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:30.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:30.948 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:30.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:30.948 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:30.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:30.948 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:30.947+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:30.948 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:30.947+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:31.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:31.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:31.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:31.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:31.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:31.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:31.061 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:31.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:31.237 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:31.239+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:31.237 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:31.239+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:31.238 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:31.239+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:31.238 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:31.239+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:31.677 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:31.675+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:31.677 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:31.675+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:31.677 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:31.675+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:31.678 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:31.675+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:31.911 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:31.907+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:31.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:31.907+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:31.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:31.907+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:31.913 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:31.907+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:31.913 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:31.907+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:32.106 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:32.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:32.106 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:32.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:32.106 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:32.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:32.107 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:32.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:32.190 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:32.191+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:32.190 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:32.191+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:32.191 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:32.191+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:32.191 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:32.191+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:32.697 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:32.695+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:32.697 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:32.695+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:32.697 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:32.695+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:32.698 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:32.695+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:32.859 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:32.859+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:32.859 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:32.859+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:32.860 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:32.859+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:32.860 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:32.859+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:32.860 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:32.859+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:33.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:33.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:33.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:33.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:33.129 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:33.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:33.129 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:33.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:33.200 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:33.203+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:33.201 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:33.203+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:33.201 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:33.203+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:33.201 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:33.203+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:33.403 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:03:33.403+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2624 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:03:33.726 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:33.723+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:33.727 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:33.723+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:33.727 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:33.723+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:33.727 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:33.723+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:33.903 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:33.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:33.903 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:33.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:33.903 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:33.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:33.904 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:33.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:33.904 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:33.903+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:34.079 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:34.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:34.079 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:34.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:34.079 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:34.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:34.080 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:34.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:34.192 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:34.191+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:34.192 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:34.191+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:34.192 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:34.191+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:34.193 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:34.191+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:34.691 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:34.691+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:34.691 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:34.691+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:34.691 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:34.691+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:34.691 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:34.691+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:34.888 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:34.887+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:34.888 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:34.887+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:34.888 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:34.887+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:34.889 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:34.887+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:34.889 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:34.887+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:35.089 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:35.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:35.090 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:35.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:35.090 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:35.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:35.090 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:35.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:35.192 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:35.191+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:35.192 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:35.191+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:35.192 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:35.191+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:35.193 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:35.191+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:35.660 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:35.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:35.660 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:35.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:35.660 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:35.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:35.660 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:35.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:35.869 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:35.867+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:35.869 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:35.867+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:35.869 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:35.867+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:35.869 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:35.867+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:35.870 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:35.867+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:36.093 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:36.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:36.093 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:36.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:36.093 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:36.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:36.094 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:36.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:36.169 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:36.171+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:36.170 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:36.171+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:36.170 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:36.171+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:36.170 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:36.171+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:36.679 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:36.679+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:36.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:36.679+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:36.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:36.679+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:36.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:36.679+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:36.870 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:36.871+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:36.870 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:36.871+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:36.870 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:36.871+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:36.870 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:36.871+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:36.871 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:36.871+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:37.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:37.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:37.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:37.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:37.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:37.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:37.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:37.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:37.138 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:37.139+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:37.138 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:37.139+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:37.139 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:37.139+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:37.139 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:37.139+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:37.667 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:37.667+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:37.668 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:37.667+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:37.668 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:37.667+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:37.668 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:37.667+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:37.906 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:37.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:37.907 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:37.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:37.907 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:37.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:37.908 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:37.903+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:37.908 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:37.903+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:38.133 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:38.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:38.134 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:38.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:38.134 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:38.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:38.134 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:38.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:38.137 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:38.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:38.137 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:38.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:38.137 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:38.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:38.138 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:38.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:38.405 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:03:38.403+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2659 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:03:38.698 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:38.695+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:38.699 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:38.695+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:38.699 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:38.695+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:38.699 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:38.695+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:38.875 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:38.875+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:38.876 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:38.875+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:38.876 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:38.875+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:38.877 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:38.875+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:38.877 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:38.875+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:39.176 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:39.175+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:39.176 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:39.175+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:39.176 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:39.175+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:39.176 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:39.175+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:39.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:39.175+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:39.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:39.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:39.180 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:39.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:39.180 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:39.179+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:39.692 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:39.691+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:39.692 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:39.691+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:39.693 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:39.691+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:39.693 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:39.691+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:39.853 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:39.851+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:39.854 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:39.851+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:39.854 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:39.851+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:39.854 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:39.851+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:39.854 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:39.851+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:40.169 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:40.167+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:40.169 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:40.167+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:40.170 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:40.167+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:40.170 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:40.167+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:40.209 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:40.211+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:40.209 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:40.211+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:40.209 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:40.211+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:40.209 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:40.211+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:40.643 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:40.643+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:40.643 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:40.643+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:40.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:40.643+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:40.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:40.643+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:40.871 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:40.871+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:40.871 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:40.871+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:40.871 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:40.871+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:40.872 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:40.871+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:40.872 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:40.871+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:41.172 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:41.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:41.172 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:41.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:41.172 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:41.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:41.173 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:41.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:41.181 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:41.183+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:41.182 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:41.183+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:41.182 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:41.183+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:41.182 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:41.183+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:41.687 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:41.687+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:41.687 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:41.687+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:41.688 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:41.687+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:41.688 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:41.687+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:41.860 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:41.859+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:41.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:41.859+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:41.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:41.859+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:41.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:41.859+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:41.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:41.859+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:42.134 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:42.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:42.134 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:42.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:42.134 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:42.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:42.135 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:42.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:42.158 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:42.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:42.158 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:42.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:42.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:42.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:42.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:42.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:42.705 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:42.707+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:42.705 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:42.707+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:42.705 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:42.707+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:42.706 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:42.707+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:42.854 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:42.855+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:42.855 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:42.855+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:42.855 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:42.855+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:42.855 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:42.855+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:42.855 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:42.855+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:43.122 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:43.119+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:43.122 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:43.119+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:43.123 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:43.119+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:43.123 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:43.119+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:43.130 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:43.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:43.130 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:43.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:43.130 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:43.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:43.130 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:43.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:43.406 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:03:43.407+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2694 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:03:43.691 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:43.691+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:43.691 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:43.691+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:43.691 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:43.691+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:43.692 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:43.691+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:43.811 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:43.807+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:43.811 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:43.807+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:43.811 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:43.807+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:43.812 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:43.807+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:43.812 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:43.807+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:44.099 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:44.099+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:44.099 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:44.099+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:44.100 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:44.099+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:44.100 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:44.099+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:44.163 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:44.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:44.163 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:44.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:44.163 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:44.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:44.164 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:44.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:44.653 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:44.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:44.653 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:44.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:44.654 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:44.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:44.654 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:44.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:44.860 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:44.859+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:44.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:44.859+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:44.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:44.859+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:44.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:44.859+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:44.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:44.859+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:45.113 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:45.111+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:45.114 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:45.111+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:45.114 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:45.111+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:45.114 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:45.111+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:45.140 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:45.139+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:45.141 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:45.139+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:45.141 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:45.139+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:45.141 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:45.139+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:45.701 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:45.699+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:45.701 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:45.699+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:45.702 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:45.699+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:45.702 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:45.699+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:45.897 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:45.895+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:45.897 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:45.895+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:45.897 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:45.895+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:45.898 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:45.895+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:45.898 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:45.895+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:46.152 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:46.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:46.153 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:46.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:46.153 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:46.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:46.153 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:46.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:46.162 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:46.163+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:46.162 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:46.163+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:46.163 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:46.163+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:46.163 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:46.163+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:46.713 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:46.711+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:46.714 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:46.711+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:46.714 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:46.711+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:46.714 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:46.711+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:46.857 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:46.855+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:46.858 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:46.855+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:46.858 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:46.855+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:46.858 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:46.855+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:46.858 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:46.855+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:47.119 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:47.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:47.120 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:47.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:47.120 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:47.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:47.120 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:47.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:47.168 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:47.167+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:47.168 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:47.167+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:47.168 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:47.167+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:47.168 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:47.167+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:47.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:47.683+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:47.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:47.683+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:47.681 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:47.683+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:47.681 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:47.683+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:47.863 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:47.863+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:47.864 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:47.863+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:47.864 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:47.863+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:47.864 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:47.863+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:47.864 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:47.863+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:48.078 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:48.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:48.079 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:48.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:48.079 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:48.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:48.079 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:48.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:48.188 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:48.187+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:48.188 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:48.187+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:48.188 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:48.187+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:48.189 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:48.187+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:48.408 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:03:48.407+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2729 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:03:48.647 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:48.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:48.648 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:48.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:48.648 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:48.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:48.648 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:48.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:48.885 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:48.882+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:48.885 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:48.882+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:48.885 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:48.882+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:48.885 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:48.882+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:48.886 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:48.882+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:49.046 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:49.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:49.047 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:49.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:49.047 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:49.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:49.047 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:49.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:49.155 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:49.154+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:49.155 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:49.154+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:49.156 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:49.154+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:49.157 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:49.154+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:49.643 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:49.642+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:49.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:49.642+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:49.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:49.642+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:49.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:49.642+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:49.911 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:49.910+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:49.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:49.910+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:49.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:49.910+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:49.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:49.910+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:49.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:49.910+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:50.074 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:50.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:50.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:50.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:50.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:50.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:50.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:50.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:50.122 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:50.118+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:50.123 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:50.118+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:50.123 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:50.118+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:50.123 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:50.118+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:50.601 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:50.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:50.601 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:50.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:50.601 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:50.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:50.602 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:50.602+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:50.901 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:50.902+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:50.902 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:50.902+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:50.902 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:50.902+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:50.902 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:50.902+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:50.903 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:50.902+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:51.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:51.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:51.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:51.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:51.077 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:51.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:51.077 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:51.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:51.083 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:51.082+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:51.083 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:51.082+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:51.084 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:51.082+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:51.084 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:51.082+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:51.613 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:51.610+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:51.614 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:51.610+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:51.614 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:51.610+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:51.614 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:51.610+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:51.937 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:51.934+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:51.937 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:51.934+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:51.938 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:51.934+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:51.938 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:51.934+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:51.938 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:51.934+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:52.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:52.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:52.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:52.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:52.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:52.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:52.054 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:52.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:52.126 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:52.126+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:52.126 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:52.126+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:52.126 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:52.126+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:52.127 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:52.126+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:52.661 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:52.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:52.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:52.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:52.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:52.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:52.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:52.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:52.975 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:52.970+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:52.975 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:52.970+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:52.975 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:52.970+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:52.975 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:52.970+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:52.976 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:52.970+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:53.042 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:53.042+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:53.042 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:53.042+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:53.042 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:53.042+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:53.043 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:53.042+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:53.108 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:53.106+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:53.108 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:53.106+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:53.108 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:53.106+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:53.109 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:53.106+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:53.409 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:03:53.406+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2765 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:03:53.678 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:53.674+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:53.678 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:53.674+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:53.679 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:53.674+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:53.679 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:53.674+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:54.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:54.014+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:54.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:54.014+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:54.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:54.014+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:54.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:54.014+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:54.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:54.018+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:54.080 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:54.078+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:54.081 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:54.078+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:54.081 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:54.078+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:54.081 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:54.078+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:54.128 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:54.130+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:54.129 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:54.130+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:54.129 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:54.130+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:54.129 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:54.130+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:54.719 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:54.718+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:54.720 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:54.718+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:54.720 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:54.718+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:54.720 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:54.718+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:55.058 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:55.058+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:55.058 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:55.058+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:55.058 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:55.058+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:55.058 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:55.058+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:55.059 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:55.058+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:55.069 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:55.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:55.069 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:55.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:55.070 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:55.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:55.070 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:55.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:55.116 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:55.118+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:55.117 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:55.118+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:55.117 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:55.118+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:55.118 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:55.118+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:55.732 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:55.734+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:55.733 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:55.734+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:55.733 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:55.734+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:55.733 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:55.734+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:56.074 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:56.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:56.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:56.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:56.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:56.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:56.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:56.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:56.076 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:56.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:56.076 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:56.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:56.076 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:56.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:56.076 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:56.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:56.076 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:56.074+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:56.078 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:56.074+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:56.078 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:56.074+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:56.078 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:56.074+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:56.079 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:56.074+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:56.708 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:56.710+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:56.709 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:56.710+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:56.709 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:56.710+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:56.709 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:56.710+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:57.035 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:57.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:57.036 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:57.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:57.036 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:57.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:57.036 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:57.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:57.046 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:57.046+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:57.046 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:57.046+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:57.046 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:57.046+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:57.047 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:57.046+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:57.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:57.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:57.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:57.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:57.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:57.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:57.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:57.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:57.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:57.102+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:57.701 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:57.702+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:57.701 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:57.702+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:57.701 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:57.702+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:57.702 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:57.702+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:58.038 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:58.038+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:58.038 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:58.038+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:58.038 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:58.038+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:58.039 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:58.038+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:58.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:58.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:58.064 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:58.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:58.064 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:58.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:58.064 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:58.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:58.146 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:58.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:58.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:58.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:58.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:58.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:58.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:58.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:58.147 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:58.146+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:58.410 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:03:58.410+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2800 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:03:58.701 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:58.698+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:58.702 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:58.698+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:58.702 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:58.698+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:58.702 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:58.698+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:59.001 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:59.002+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:59.002 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:59.002+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:59.002 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:59.002+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:59.002 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:03:59.002+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:03:59.023 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:59.022+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:59.024 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:59.022+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:59.024 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:59.022+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:59.024 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:03:59.022+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:03:59.133 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:59.134+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:59.133 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:59.134+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:59.133 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:59.134+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:59.134 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:59.134+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:03:59.134 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:03:59.134+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:03:59.728 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:59.730+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:59.728 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:59.730+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:59.728 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:59.730+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:03:59.729 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:03:59.730+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:00.011 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:00.006+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:00.011 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:00.006+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:00.012 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:00.006+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:00.012 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:00.006+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:00.026 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:00.022+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:00.026 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:00.022+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:00.027 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:00.022+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:00.027 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:00.022+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:00.164 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:00.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:00.165 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:00.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:00.165 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:00.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:00.165 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:00.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:00.165 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:00.162+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:00.689 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:00.690+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:00.689 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:00.690+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:00.690 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:00.690+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:00.690 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:00.690+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:01.002 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:01.002+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:01.002 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:01.002+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:01.003 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:01.002+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:01.003 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:01.002+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:01.061 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:01.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:01.061 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:01.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:01.061 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:01.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:01.062 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:01.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:01.115 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:01.110+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:01.115 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:01.110+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:01.116 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:01.110+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:01.116 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:01.110+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:01.116 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:01.110+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:01.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:01.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:01.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:01.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:01.663 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:01.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:01.663 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:01.662+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:01.972 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:01.970+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:01.973 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:01.970+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:01.973 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:01.970+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:01.973 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:01.970+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:02.067 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:02.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:02.067 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:02.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:02.068 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:02.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:02.068 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:02.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:02.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:02.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:02.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:02.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:02.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:02.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:02.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:02.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:02.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:02.102+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:02.679 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:02.674+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:02.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:02.674+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:02.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:02.674+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:02.680 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:02.674+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:03.015 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:03.014+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:03.016 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:03.014+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:03.016 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:03.014+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:03.017 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:03.014+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:03.041 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:03.042+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:03.042 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:03.042+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:03.042 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:03.042+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:03.042 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:03.042+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:03.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:03.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:03.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:03.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:03.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:03.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:03.103 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:03.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:03.103 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:03.098+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:03.411 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:04:03.410+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2837 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:04:03.693 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:03.694+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:03.694 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:03.694+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:03.694 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:03.694+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:03.694 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:03.694+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:04.006 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:04.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:04.007 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:04.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:04.008 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:04.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:04.008 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:04.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:04.031 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:04.030+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:04.031 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:04.030+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:04.031 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:04.030+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:04.032 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:04.030+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:04.083 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:04.082+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:04.084 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:04.082+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:04.084 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:04.082+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:04.084 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:04.082+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:04.085 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:04.082+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:04.742 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:04.734+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:04.742 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:04.734+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:04.742 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:04.734+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:04.743 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:04.734+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:05.015 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:05.014+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:05.015 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:05.014+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:05.016 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:05.014+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:05.016 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:05.014+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:05.030 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:05.030+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:05.030 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:05.030+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:05.031 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:05.030+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:05.031 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:05.030+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:05.064 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:05.066+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:05.065 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:05.066+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:05.065 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:05.066+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:05.065 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:05.066+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:05.065 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:05.066+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:05.730 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:05.730+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:05.730 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:05.730+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:05.731 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:05.730+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:05.731 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:05.730+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:06.004 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:06.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:06.004 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:06.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:06.004 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:06.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:06.005 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:06.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:06.027 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:06.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:06.027 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:06.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:06.028 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:06.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:06.028 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:06.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:06.028 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:06.022+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:06.055 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:06.054+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:06.056 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:06.054+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:06.056 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:06.054+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:06.056 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:06.054+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:06.765 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:06.766+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:06.766 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:06.766+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:06.766 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:06.766+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:06.766 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:06.766+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:07.035 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:07.034+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:07.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:07.034+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:07.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:07.034+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:07.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:07.034+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:07.036 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:07.034+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:07.045 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:07.046+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:07.045 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:07.046+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:07.046 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:07.046+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:07.046 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:07.046+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:07.047 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:07.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:07.047 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:07.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:07.047 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:07.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:07.047 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:07.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:07.793 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:07.790+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:07.794 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:07.790+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:07.794 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:07.790+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:07.794 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:07.790+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:07.994 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:07.994+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:07.994 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:07.994+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:07.994 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:07.994+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:07.995 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:07.994+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:07.995 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:07.994+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:08.022 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:08.022+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:08.023 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:08.022+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:08.023 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:08.022+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:08.023 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:08.022+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:08.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:08.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:08.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:08.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:08.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:08.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:08.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:08.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:08.413 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:04:08.410+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2871 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:04:08.842 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:08.842+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:08.843 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:08.842+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:08.843 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:08.842+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:08.843 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:08.842+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:09.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:09.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:09.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:09.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:09.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:09.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:09.026 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:09.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:09.027 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:09.022+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:09.066 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:09.058+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:09.066 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:09.058+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:09.066 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:09.058+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:09.066 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:09.058+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:09.074 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:09.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:09.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:09.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:09.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:09.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:09.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:09.070+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:09.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:09.890+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:09.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:09.890+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:09.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:09.890+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:09.893 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:09.890+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:10.020 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:10.018+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:10.021 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:10.018+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:10.021 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:10.018+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:10.021 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:10.018+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:10.050 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:10.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:10.050 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:10.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:10.051 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:10.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:10.051 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:10.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:10.066 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:10.062+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:10.066 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:10.062+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:10.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:10.062+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:10.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:10.062+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:10.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:10.062+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:10.872 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:10.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:10.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:10.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:10.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:10.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:10.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:10.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:10.984 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:10.986+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:10.985 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:10.986+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:10.985 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:10.986+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:10.985 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:10.986+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:11.086 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:11.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:11.087 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:11.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:11.087 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:11.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:11.088 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:11.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:11.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:11.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:11.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:11.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:11.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:11.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:11.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:11.102+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:11.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:11.102+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:11.890 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:11.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:11.890 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:11.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:11.891 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:11.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:11.891 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:11.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:11.958 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:11.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:11.959 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:11.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:11.959 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:11.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:11.959 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:11.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:12.043 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:12.038+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:12.044 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:12.038+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:12.044 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:12.038+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:12.044 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:12.038+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:12.091 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:12.090+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:12.091 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:12.090+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:12.091 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:12.090+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:12.091 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:12.090+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:12.092 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:12.090+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:12.852 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:12.850+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:12.853 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:12.850+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:12.853 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:12.850+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:12.853 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:12.850+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:12.967 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:12.966+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:12.967 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:12.966+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:12.967 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:12.966+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:12.967 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:12.966+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:13.042 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:13.042+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:13.042 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:13.042+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:13.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:13.042+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:13.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:13.042+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:13.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:13.042+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:13.057 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:13.058+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:13.058 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:13.058+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:13.058 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:13.058+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:13.058 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:13.058+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:13.414 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:04:13.414+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2908 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:04:13.876 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:13.878+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:13.877 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:13.878+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:13.877 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:13.878+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:13.877 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:13.878+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:13.949 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:13.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:13.949 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:13.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:13.949 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:13.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:13.950 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:13.950+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:14.075 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:14.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:14.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:14.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:14.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:14.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:14.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:14.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:14.091 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:14.090+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:14.091 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:14.090+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:14.091 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:14.090+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:14.091 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:14.090+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:14.092 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:14.090+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:14.870 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:14.870+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:14.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:14.870+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:14.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:14.870+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:14.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:14.870+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:14.967 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:14.966+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:14.967 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:14.966+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:14.967 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:14.966+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:14.967 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:14.966+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:15.076 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:15.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:15.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:15.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:15.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:15.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:15.194 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:15.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:15.194 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:15.074+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:15.195 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:15.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:15.195 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:15.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:15.195 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:15.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:15.195 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:15.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:15.911 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:15.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:15.911 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:15.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:15.911 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:15.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:15.912 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:15.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:15.942 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:15.934+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:15.942 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:15.934+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:15.943 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:15.934+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:15.943 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:15.934+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:16.076 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:16.078+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:16.076 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:16.078+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:16.076 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:16.078+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:16.077 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:16.078+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:16.077 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:16.078+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:16.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:16.126+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:16.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:16.126+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:16.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:16.126+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:16.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:16.126+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:16.896 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:16.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:16.897 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:16.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:16.897 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:16.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:16.897 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:16.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:16.901 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:16.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:16.901 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:16.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:16.901 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:16.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:16.901 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:16.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:17.069 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:17.070+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:17.069 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:17.070+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:17.070 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:17.070+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:17.070 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:17.070+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:17.070 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:17.070+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:17.142 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:17.142+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:17.142 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:17.142+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:17.142 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:17.142+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:17.142 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:17.142+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:17.859 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:17.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:17.859 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:17.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:17.859 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:17.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:17.859 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:17.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:17.860 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:17.862+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:17.860 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:17.862+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:17.860 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:17.862+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:17.861 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:17.862+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:18.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:18.070+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:18.068 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:18.070+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:18.068 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:18.070+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:18.068 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:18.070+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:18.069 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:18.070+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:18.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:18.130+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:18.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:18.130+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:18.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:18.130+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:18.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:18.130+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:18.415 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:04:18.414+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2943 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:04:18.813 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:18.814+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:18.814 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:18.814+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:18.814 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:18.814+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:18.814 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:18.814+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:18.896 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:18.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:18.897 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:18.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:18.897 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:18.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:18.897 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:18.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:19.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:19.054+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:19.054 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:19.054+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:19.054 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:19.054+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:19.054 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:19.054+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:19.054 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:19.054+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:19.150 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:19.150+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:19.150 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:19.150+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:19.151 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:19.150+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:19.151 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:19.150+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:19.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:19.862+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:19.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:19.862+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:19.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:19.862+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:19.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:19.862+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:19.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:19.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:19.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:19.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:19.877 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:19.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:19.877 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:19.878+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:20.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:20.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:20.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:20.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:20.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:20.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:20.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:20.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:20.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:20.022+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:20.174 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:20.174+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:20.175 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:20.174+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:20.175 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:20.174+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:20.175 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:20.174+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:20.882 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:20.882+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:20.882 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:20.882+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:20.883 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:20.882+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:20.883 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:20.882+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:20.887 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:20.890+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:20.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:20.890+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:20.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:20.890+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:20.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:20.890+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:21.063 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:21.062+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:21.063 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:21.062+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:21.063 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:21.062+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:21.064 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:21.062+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:21.064 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:21.062+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:21.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:21.178+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:21.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:21.178+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:21.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:21.178+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:21.180 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:21.178+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:21.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:21.854+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:21.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:21.854+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:21.856 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:21.854+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:21.856 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:21.854+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:21.899 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:21.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:21.899 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:21.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:21.899 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:21.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:21.900 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:21.898+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:22.023 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:22.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:22.023 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:22.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:22.024 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:22.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:22.024 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:22.022+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:22.024 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:22.026+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:22.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:22.154+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:22.160 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:22.154+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:22.160 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:22.154+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:22.160 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:22.154+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:22.894 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:22.894+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:22.895 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:22.894+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:22.895 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:22.894+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:22.895 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:22.894+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:22.927 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:22.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:22.928 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:22.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:22.928 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:22.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:22.928 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:22.930+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:22.987 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:22.990+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:22.988 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:22.990+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:22.988 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:22.990+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:22.988 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:22.990+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:22.988 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:22.990+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:23.155 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:23.158+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:23.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:23.158+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:23.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:23.158+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:23.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:23.158+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:23.416 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:04:23.418+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 2980 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:04:23.922 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:23.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:23.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:23.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:23.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:23.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:23.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:23.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:23.940 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:23.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:23.940 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:23.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:23.941 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:23.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:23.941 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:23.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:23.987 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:23.986+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:23.987 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:23.986+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:23.988 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:23.986+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:23.988 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:23.986+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:23.988 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:23.990+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:24.117 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:24.118+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:24.118 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:24.118+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:24.118 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:24.118+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:24.118 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:24.118+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:24.880 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:24.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:24.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:24.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:24.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:24.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:24.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:24.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:24.919 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:24.922+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:24.920 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:24.922+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:24.920 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:24.922+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:24.920 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:24.922+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:25.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:25.018+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:25.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:25.018+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:25.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:25.018+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:25.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:25.018+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:25.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:25.018+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:25.119 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:25.118+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:25.120 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:25.118+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:25.120 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:25.118+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:25.120 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:25.118+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:25.919 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:25.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:25.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:25.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:25.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:25.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:25.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:25.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:25.966 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:25.966+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:25.967 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:25.966+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:25.967 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:25.966+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:25.967 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:25.966+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:26.052 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:26.054+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:26.052 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:26.054+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:26.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:26.054+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:26.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:26.054+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:26.053 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:26.054+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:26.164 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:26.166+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:26.165 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:26.166+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:26.165 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:26.166+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:26.165 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:26.166+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:26.939 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:26.942+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:26.940 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:26.942+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:26.940 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:26.942+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:26.940 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:26.942+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:26.979 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:26.978+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:26.979 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:26.978+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:26.979 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:26.978+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:26.979 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:26.978+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:27.013 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:27.014+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:27.013 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:27.014+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:27.014 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:27.014+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:27.014 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:27.014+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:27.015 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:27.014+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:27.180 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:27.182+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:27.181 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:27.182+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:27.181 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:27.182+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:27.181 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:27.182+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:27.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:27.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:27.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:27.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:27.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:27.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:27.914 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:27.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:27.973 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:27.974+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:27.974 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:27.974+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:27.974 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:27.974+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:27.974 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:27.974+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:27.974 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:27.974+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:28.001 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:28.002+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:28.002 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:28.002+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:28.002 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:28.002+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:28.002 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:28.002+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:28.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:28.178+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:28.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:28.178+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:28.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:28.178+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:28.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:28.178+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:28.418 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:04:28.418+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3014 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:04:28.952 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:28.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:28.952 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:28.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:28.952 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:28.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:28.953 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:28.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:29.001 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:28.998+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:29.002 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:28.998+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:29.002 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:28.998+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:29.002 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:28.998+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:29.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:29.002+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:29.034 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:29.034+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:29.034 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:29.034+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:29.034 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:29.034+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:29.035 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:29.034+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:29.221 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:29.222+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:29.221 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:29.222+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:29.222 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:29.222+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:29.222 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:29.222+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:29.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:29.906+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:29.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:29.906+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:29.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:29.906+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:29.905 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:29.906+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:30.002 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:30.002+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:30.002 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:30.002+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:30.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:30.002+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:30.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:30.002+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:30.003 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:30.002+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:30.067 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:30.066+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:30.067 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:30.066+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:30.067 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:30.066+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:30.068 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:30.066+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:30.231 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:30.230+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:30.231 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:30.230+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:30.232 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:30.230+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:30.232 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:30.230+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:30.863 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:30.866+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:30.864 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:30.866+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:30.864 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:30.866+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:30.864 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:30.866+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:30.982 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:30.982+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:30.982 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:30.982+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:30.983 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:30.982+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:30.983 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:30.982+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:30.983 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:30.982+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:31.080 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:31.082+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:31.080 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:31.082+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:31.080 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:31.082+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:31.081 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:31.082+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:31.272 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:31.270+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:31.272 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:31.270+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:31.273 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:31.270+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:31.273 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:31.270+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:31.850 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:31.850+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:31.851 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:31.850+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:31.851 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:31.850+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:31.851 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:31.850+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:31.962 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:31.962+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:31.963 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:31.962+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:31.963 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:31.962+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:31.963 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:31.962+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:31.963 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:31.962+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:32.079 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:32.078+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:32.080 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:32.078+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:32.080 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:32.078+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:32.080 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:32.078+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:32.238 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:32.238+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:32.239 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:32.238+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:32.239 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:32.238+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:32.239 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:32.238+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:32.891 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:32.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:32.892 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:32.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:32.892 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:32.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:32.892 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:32.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:32.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:32.914+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:32.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:32.914+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:32.913 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:32.914+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:32.913 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:32.914+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:32.913 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:32.914+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:33.062 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:33.062+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:33.062 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:33.062+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:33.063 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:33.062+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:33.063 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:33.062+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:33.254 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:33.250+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:33.255 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:33.250+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:33.255 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:33.250+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:33.255 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:33.250+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:33.419 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:04:33.418+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3051 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:04:33.890 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:33.890+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:33.890 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:33.890+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:33.891 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:33.890+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:33.891 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:33.890+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:33.891 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:33.890+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:33.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:33.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:33.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:33.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:33.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:33.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:33.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:33.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:34.101 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:34.102+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:34.102 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:34.102+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:34.102 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:34.102+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:34.103 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:34.102+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:34.258 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:34.258+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:34.259 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:34.258+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:34.259 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:34.258+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:34.259 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:34.258+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:34.917 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:34.918+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:34.918 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:34.918+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:34.918 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:34.918+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:34.918 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:34.918+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:34.932 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:34.934+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:34.932 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:34.934+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:34.932 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:34.934+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:34.933 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:34.934+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:34.933 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:34.934+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:35.145 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:35.146+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:35.145 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:35.146+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:35.146 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:35.146+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:35.146 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:35.146+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:35.271 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:35.274+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:35.271 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:35.274+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:35.271 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:35.274+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:35.272 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:35.274+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:35.938 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:35.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:35.939 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:35.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:35.939 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:35.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:35.939 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:35.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:35.952 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:35.954+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:35.952 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:35.954+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:35.953 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:35.954+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:35.953 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:35.954+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:35.953 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:35.954+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:36.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:36.170+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:36.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:36.170+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:36.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:36.170+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:36.174 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:36.170+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:36.261 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:36.261+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:36.262 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:36.261+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:36.262 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:36.261+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:36.262 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:36.261+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:36.902 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:36.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:36.903 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:36.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:36.903 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:36.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:36.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:36.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:36.913 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:36.913+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:36.913 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:36.913+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:36.913 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:36.913+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:36.914 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:36.913+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:36.914 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:36.913+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:37.210 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:37.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:37.211 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:37.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:37.211 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:37.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:37.211 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:37.209+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:37.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:37.289+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:37.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:37.289+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:37.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:37.289+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:37.292 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:37.289+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:37.874 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:37.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:37.874 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:37.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:37.874 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:37.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:37.874 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:37.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:37.897 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:37.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:37.897 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:37.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:37.898 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:37.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:37.898 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:37.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:37.898 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:37.897+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:38.194 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:38.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:38.195 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:38.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:38.195 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:38.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:38.195 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:38.193+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:38.265 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:38.265+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:38.266 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:38.265+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:38.266 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:38.265+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:38.266 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:38.265+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:38.426 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:04:38.421+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3085 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:04:38.834 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:38.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:38.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:38.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:38.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:38.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:38.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:38.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:38.872 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:38.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:38.872 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:38.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:38.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:38.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:38.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:38.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:38.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:38.873+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:39.170 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:39.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:39.170 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:39.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:39.170 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:39.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:39.171 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:39.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:39.246 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:39.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:39.246 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:39.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:39.247 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:39.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:39.247 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:39.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:39.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:39.789+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:39.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:39.789+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:39.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:39.789+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:39.790 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:39.789+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:39.866 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:39.865+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:39.866 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:39.865+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:39.866 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:39.865+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:39.867 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:39.865+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:39.867 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:39.865+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:40.188 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:40.189+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:40.189 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:40.189+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:40.189 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:40.189+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:40.189 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:40.189+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:40.254 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:40.253+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:40.254 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:40.253+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:40.255 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:40.253+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:40.255 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:40.253+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:40.833 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:40.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:40.833 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:40.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:40.833 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:40.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:40.833 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:40.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:40.908 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:40.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:40.909 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:40.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:40.909 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:40.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:40.909 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:40.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:40.910 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:40.909+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:41.208 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:41.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:41.209 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:41.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:41.209 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:41.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:41.209 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:41.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:41.216 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:41.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:41.217 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:41.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:41.217 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:41.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:41.217 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:41.217+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:41.833 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:41.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:41.833 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:41.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:41.834 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:41.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:41.834 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:41.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:41.910 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:41.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:41.911 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:41.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:41.911 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:41.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:41.911 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:41.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:41.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:41.909+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:42.187 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:42.185+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:42.187 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:42.185+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:42.188 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:42.185+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:42.188 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:42.185+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:42.234 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:42.233+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:42.235 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:42.233+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:42.235 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:42.233+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:42.235 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:42.233+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:42.821 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:42.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:42.822 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:42.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:42.822 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:42.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:42.822 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:42.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:42.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:42.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:42.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:42.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:42.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:42.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:42.874 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:42.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:42.874 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:42.873+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:43.225 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:43.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:43.225 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:43.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:43.226 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:43.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:43.226 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:43.225+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:43.247 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:43.249+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:43.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:43.249+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:43.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:43.249+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:43.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:43.249+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:43.422 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:04:43.421+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3122 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:04:43.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:43.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:43.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:43.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:43.849 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:43.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:43.849 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:43.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:43.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:43.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:43.873 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:43.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:43.874 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:43.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:43.874 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:43.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:43.874 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:43.873+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:44.233 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:44.233+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:44.234 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:44.233+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:44.234 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:44.233+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:44.234 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:44.233+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:44.247 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:44.249+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:44.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:44.249+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:44.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:44.249+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:44.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:44.249+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:44.812 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:44.813+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:44.813 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:44.813+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:44.813 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:44.813+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:44.813 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:44.813+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:44.887 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:44.889+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:44.887 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:44.889+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:44.888 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:44.889+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:44.888 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:44.889+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:44.889 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:44.889+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:45.251 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:45.249+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:45.251 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:45.249+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:45.252 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:45.249+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:45.252 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:45.249+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:45.269 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:45.269+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:45.270 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:45.269+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:45.270 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:45.269+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:45.270 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:45.269+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:45.828 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:45.829+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:45.829 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:45.829+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:45.829 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:45.829+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:45.829 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:45.829+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:45.878 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:45.877+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:45.879 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:45.877+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:45.879 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:45.877+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:45.879 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:45.877+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:45.879 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:45.877+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:46.241 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:46.241+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:46.242 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:46.241+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:46.242 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:46.241+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:46.242 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:46.241+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:46.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:46.297+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:46.297 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:46.297+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:46.297 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:46.297+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:46.298 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:46.297+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:46.797 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:46.793+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:46.797 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:46.793+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:46.798 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:46.793+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:46.798 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:46.793+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:46.928 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:46.929+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:46.928 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:46.929+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:46.929 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:46.929+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:46.929 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:46.929+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:46.929 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:46.929+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:47.210 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:47.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:47.211 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:47.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:47.211 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:47.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:47.211 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:47.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:47.309 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:47.309+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:47.310 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:47.309+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:47.310 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:47.309+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:47.310 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:47.309+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:47.837 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:47.837+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:47.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:47.837+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:47.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:47.837+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:47.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:47.837+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:47.880 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:47.881+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:47.880 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:47.881+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:47.880 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:47.881+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:47.881 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:47.881+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:47.881 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:47.881+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:48.200 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:48.201+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:48.200 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:48.201+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:48.200 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:48.201+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:48.200 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:48.201+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:48.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:48.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:48.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:48.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:48.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:48.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:48.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:48.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:48.423 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:04:48.421+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3156 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:04:48.880 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:48.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:48.880 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:48.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:48.880 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:48.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:48.880 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:48.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:48.920 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:48.921+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:48.921 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:48.921+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:48.921 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:48.921+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:48.921 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:48.921+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:48.921 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:48.921+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:49.164 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:49.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:49.165 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:49.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:49.165 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:49.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:49.165 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:49.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:49.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:49.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:49.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:49.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:49.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:49.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:49.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:49.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:49.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:49.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:49.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:49.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:49.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:49.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:49.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:49.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:49.923 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:49.925+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:49.923 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:49.925+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:49.923 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:49.925+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:49.924 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:49.925+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:49.924 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:49.925+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:50.194 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:50.193+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:50.195 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:50.193+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:50.195 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:50.193+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:50.195 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:50.193+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:50.369 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:50.369+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:50.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:50.369+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:50.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:50.369+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:50.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:50.369+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:50.894 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:50.893+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:50.895 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:50.893+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:50.895 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:50.893+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:50.895 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:50.893+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:50.945 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:50.945+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:50.945 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:50.945+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:50.946 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:50.945+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:50.946 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:50.945+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:50.946 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:50.945+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:51.210 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:51.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:51.210 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:51.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:51.211 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:51.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:51.211 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:51.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:51.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:51.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:51.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:51.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:51.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:51.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:51.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:51.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:51.934 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:51.933+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:51.934 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:51.933+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:51.935 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:51.933+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:51.935 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:51.933+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:51.935 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:51.933+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:51.935 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:51.933+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:51.935 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:51.933+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:51.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:51.933+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:51.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:51.933+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:52.252 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:52.253+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:52.253 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:52.253+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:52.253 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:52.253+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:52.253 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:52.253+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:52.385 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:52.385+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:52.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:52.385+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:52.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:52.385+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:52.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:52.385+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:52.896 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:52.897+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:52.896 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:52.897+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:52.897 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:52.897+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:52.897 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:52.897+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:52.977 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:52.977+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:52.978 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:52.977+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:52.978 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:52.977+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:52.978 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:52.977+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:52.978 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:52.977+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:53.244 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:53.241+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:53.244 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:53.241+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:53.244 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:53.241+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:53.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:53.241+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:53.345 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:53.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:53.345 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:53.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:53.345 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:53.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:53.345 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:53.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:53.424 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:04:53.425+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3193 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:04:53.863 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:53.865+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:53.864 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:53.865+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:53.864 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:53.865+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:53.864 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:53.865+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:53.949 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:53.949+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:53.950 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:53.949+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:53.950 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:53.949+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:53.950 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:53.949+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:53.950 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:53.949+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:54.290 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:54.289+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:54.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:54.289+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:54.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:54.289+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:54.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:54.289+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:54.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:54.337+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:54.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:54.337+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:54.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:54.337+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:54.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:54.337+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:54.877 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:54.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:54.877 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:54.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:54.878 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:54.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:54.878 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:54.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:54.926 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:54.925+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:54.926 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:54.925+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:54.926 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:54.925+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:54.926 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:54.925+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:54.927 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:54.925+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:55.327 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:55.329+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:55.328 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:55.329+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:55.328 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:55.329+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:55.328 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:55.329+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:55.329 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:55.329+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:55.329 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:55.329+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:55.329 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:55.329+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:55.330 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:55.329+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:55.911 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:55.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:55.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:55.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:55.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:55.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:55.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:55.909+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:55.912 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:55.913+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:55.927 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:55.925+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:55.927 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:55.925+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:55.927 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:55.925+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:55.927 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:55.925+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:56.311 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:56.309+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:56.311 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:56.309+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:56.311 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:56.309+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:56.311 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:56.309+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:56.377 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:56.377+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:56.378 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:56.377+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:56.378 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:56.377+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:56.378 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:56.377+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:56.905 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:56.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:56.906 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:56.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:56.906 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:56.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:56.906 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:56.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:56.941 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:56.941+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:56.942 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:56.941+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:56.942 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:56.941+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:56.942 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:56.941+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:56.942 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:56.941+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:57.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:57.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:57.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:57.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:57.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:57.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:57.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:57.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:57.375 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:57.377+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:57.376 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:57.377+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:57.376 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:57.377+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:57.376 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:57.377+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:57.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:57.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:57.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:57.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:57.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:57.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:57.890 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:57.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:57.898 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:57.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:57.898 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:57.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:57.899 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:57.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:57.899 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:57.897+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:57.899 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:57.897+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:58.337 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:58.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:58.337 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:58.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:58.337 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:58.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:58.338 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:58.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:58.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:58.353+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:58.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:58.353+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:58.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:58.353+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:58.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:58.353+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:58.425 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:04:58.425+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3228 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:04:58.935 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:58.933+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:58.935 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:58.933+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:58.935 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:58.933+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:58.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:58.933+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:58.936 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:58.937+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:58.937 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:58.937+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:58.937 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:58.937+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:58.937 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:58.937+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:58.937 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:58.937+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:04:59.314 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:59.313+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:59.314 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:59.313+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:59.315 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:59.313+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:59.315 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:04:59.313+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:04:59.375 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:59.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:59.375 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:59.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:59.375 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:59.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:59.375 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:04:59.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:04:59.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:59.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:59.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:59.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:59.905 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:59.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:59.905 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:04:59.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:04:59.934 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:59.933+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:59.935 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:59.933+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:59.935 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:59.933+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:59.935 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:59.933+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:04:59.935 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:04:59.933+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:00.278 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:00.277+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:00.279 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:00.277+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:00.279 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:00.277+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:00.279 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:00.277+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:00.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:00.381+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:00.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:00.381+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:00.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:00.381+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:00.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:00.381+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:00.888 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:00.889+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:00.888 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:00.889+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:00.889 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:00.889+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:00.889 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:00.889+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:00.889 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:00.889+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:00.907 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:00.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:00.907 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:00.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:00.908 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:00.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:00.908 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:00.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:01.238 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:01.237+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:01.238 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:01.237+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:01.238 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:01.237+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:01.239 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:01.237+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:01.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:01.337+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:01.339 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:01.337+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:01.339 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:01.337+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:01.339 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:01.337+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:01.841 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:01.841+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:01.841 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:01.841+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:01.841 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:01.841+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:01.842 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:01.841+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:01.842 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:01.841+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:01.858 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:01.857+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:01.858 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:01.857+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:01.858 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:01.857+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:01.859 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:01.857+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:02.223 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:02.225+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:02.223 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:02.225+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:02.224 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:02.225+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:02.224 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:02.225+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:02.295 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:02.297+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:02.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:02.297+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:02.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:02.297+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:02.296 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:02.297+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:02.843 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:02.841+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:02.843 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:02.841+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:02.843 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:02.841+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:02.844 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:02.841+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:02.844 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:02.841+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:02.846 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:02.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:02.846 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:02.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:02.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:02.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:02.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:02.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:03.176 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:03.177+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:03.176 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:03.177+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:03.176 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:03.177+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:03.177 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:03.177+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:03.259 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:03.257+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:03.259 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:03.257+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:03.259 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:03.257+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:03.260 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:03.257+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:03.427 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:05:03.425+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3265 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:05:03.834 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:03.833+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:03.834 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:03.833+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:03.835 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:03.833+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:03.835 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:03.833+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:03.835 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:03.833+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:03.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:03.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:03.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:03.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:03.890 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:03.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:03.890 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:03.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:04.208 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:04.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:04.208 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:04.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:04.208 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:04.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:04.209 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:04.209+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:04.292 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:04.293+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:04.292 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:04.293+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:04.292 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:04.293+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:04.292 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:04.293+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:04.829 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:04.825+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:04.829 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:04.825+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:04.829 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:04.825+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:04.830 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:04.825+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:04.830 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:04.825+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:04.915 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:04.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:04.916 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:04.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:04.916 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:04.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:04.916 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:04.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:05.244 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:05.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:05.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:05.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:05.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:05.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:05.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:05.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:05.321 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:05.321+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:05.321 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:05.321+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:05.321 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:05.321+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:05.321 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:05.321+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:05.874 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:05.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:05.874 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:05.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:05.875 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:05.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:05.875 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:05.873+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:05.875 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:05.873+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:05.907 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:05.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:05.908 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:05.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:05.908 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:05.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:05.908 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:05.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:06.244 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:06.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:06.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:06.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:06.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:06.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:06.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:06.245+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:06.328 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:06.329+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:06.329 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:06.329+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:06.329 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:06.329+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:06.330 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:06.329+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:06.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:06.861+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:06.861 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:06.861+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:06.862 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:06.861+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:06.862 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:06.861+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:06.862 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:06.861+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:06.921 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:06.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:06.922 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:06.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:06.922 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:06.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:06.922 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:06.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:07.202 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:07.201+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:07.202 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:07.201+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:07.202 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:07.201+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:07.203 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:07.201+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:07.340 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:07.341+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:07.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:07.341+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:07.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:07.341+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:07.342 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:07.341+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:07.812 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:07.809+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:07.812 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:07.809+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:07.813 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:07.809+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:07.813 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:07.809+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:07.813 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:07.809+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:07.899 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:07.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:07.900 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:07.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:07.900 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:07.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:07.900 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:07.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:08.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:08.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:08.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:08.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:08.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:08.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:08.167 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:08.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:08.304 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:08.297+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:08.304 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:08.297+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:08.305 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:08.297+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:08.305 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:08.297+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:08.428 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:05:08.429+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3299 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:05:08.813 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:08.813+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:08.814 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:08.813+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:08.814 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:08.813+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:08.815 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:08.813+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:08.815 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:08.813+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:08.875 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:08.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:08.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:08.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:08.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:08.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:08.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:08.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:09.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:09.121+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:09.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:09.121+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:09.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:09.121+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:09.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:09.121+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:09.324 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:09.325+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:09.324 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:09.325+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:09.325 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:09.325+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:09.325 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:09.325+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:09.822 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:09.821+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:09.823 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:09.821+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:09.823 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:09.821+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:09.823 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:09.821+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:09.823 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:09.821+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:09.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:09.837+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:09.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:09.837+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:09.836 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:09.837+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:09.836 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:09.837+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:10.161 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:10.161+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:10.162 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:10.161+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:10.162 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:10.161+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:10.162 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:10.161+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:10.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:10.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:10.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:10.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:10.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:10.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:10.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:10.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:10.777 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:10.777+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:10.778 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:10.777+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:10.778 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:10.777+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:10.778 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:10.777+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:10.778 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:10.777+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:10.834 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:10.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:10.834 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:10.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:10.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:10.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:10.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:10.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:11.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:11.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:11.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:11.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:11.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:11.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:11.129 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:11.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:11.355 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:11.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:11.356 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:11.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:11.356 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:11.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:11.356 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:11.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:11.735 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:11.737+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:11.736 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:11.737+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:11.736 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:11.737+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:11.736 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:11.737+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:11.736 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:11.737+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:11.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:11.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:11.877 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:11.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:11.877 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:11.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:11.877 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:11.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:12.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:12.125+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:12.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:12.125+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:12.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:12.125+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:12.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:12.125+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:12.340 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:12.341+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:12.340 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:12.341+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:12.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:12.341+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:12.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:12.341+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:12.723 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:12.725+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.906163+0000 front 2020-05-18T21:54:16.905443+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:12.723 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:12.725+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.905661+0000 front 2020-05-18T21:54:16.905564+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:12.724 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:12.725+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.906019+0000 front 2020-05-18T21:54:16.905247+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:12.724 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:12.725+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.905878+0000 front 2020-05-18T21:54:16.905809+0000 (oldest deadline 2020-05-18T21:54:58.104508+0000) 2020-05-18T22:05:12.724 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:12.725+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:12.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:12.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:12.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:12.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:12.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:12.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:12.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:12.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:13.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:13.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:13.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:13.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:13.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:13.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:13.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:13.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:13.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:13.389+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:17.863279+0000 front 2020-05-18T21:54:17.863972+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:13.388 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:13.389+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:17.863535+0000 front 2020-05-18T21:54:17.863497+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:13.388 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:13.389+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:17.863712+0000 front 2020-05-18T21:54:17.862658+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:13.388 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:13.389+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:17.863750+0000 front 2020-05-18T21:54:17.863933+0000 (oldest deadline 2020-05-18T21:54:58.106440+0000) 2020-05-18T22:05:13.429 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:05:13.429+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3336 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:05:13.746 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:13.745+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:13.878 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:13.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:15.960552+0000 front 2020-05-18T21:54:15.960425+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:13.878 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:13.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:15.960458+0000 front 2020-05-18T21:54:15.960054+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:13.878 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:13.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:15.960638+0000 front 2020-05-18T21:54:15.960307+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:13.879 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:13.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:15.959970+0000 front 2020-05-18T21:54:15.960778+0000 (oldest deadline 2020-05-18T21:54:58.101596+0000) 2020-05-18T22:05:14.174 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:14.177+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:14.175 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:14.177+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:14.175 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:14.177+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:14.176 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:14.177+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:14.768 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:14.769+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:15.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:15.181+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 since back 2020-05-18T21:54:16.167371+0000 front 2020-05-18T21:54:16.167946+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:15.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:15.181+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 since back 2020-05-18T21:54:16.167700+0000 front 2020-05-18T21:54:16.167760+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:15.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:15.181+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 since back 2020-05-18T21:54:16.167644+0000 front 2020-05-18T21:54:16.168458+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:15.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:15.181+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 since back 2020-05-18T21:54:16.168071+0000 front 2020-05-18T21:54:16.167902+0000 (oldest deadline 2020-05-18T21:54:58.112100+0000) 2020-05-18T22:05:15.739 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:15.741+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:16.759 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:16.761+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:17.750 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:17.749+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:18.431 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:05:18.429+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3371 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:05:18.752 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:18.753+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:19.739 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:19.741+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:20.772 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:20.773+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:21.814 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:21.813+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:22.793 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:22.793+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:23.432 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:05:23.433+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3408 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:05:23.826 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:23.825+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:24.797 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:24.797+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:25.843 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:25.844+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:26.847 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:26.848+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:27.850 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:27.852+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:28.434 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:05:28.432+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3442 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:05:28.854 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:28.852+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:29.880 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:29.880+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:30.836 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:30.836+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:31.793 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:31.792+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:32.754 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:32.752+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:33.436 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:05:33.436+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3478 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:05:33.710 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:33.712+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:34.675 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:34.672+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:35.633 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:35.632+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:36.129 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:36.128+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:36.129 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:36.128+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:36.129 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:36.128+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:36.130 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:36.128+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:36.636 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:36.636+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:36.636 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:36.636+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:36.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:36.636+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:36.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:36.636+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:36.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:36.636+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:37.089 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:37.088+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:37.089 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:37.088+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:37.090 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:37.088+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:37.090 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:37.088+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:37.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:37.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:37.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:37.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:37.622 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:37.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:37.622 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:37.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:37.622 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:37.620+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:38.138 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:38.136+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:38.139 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:38.136+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:38.139 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:38.136+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:38.139 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:38.136+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:38.437 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:05:38.436+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3514 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:05:38.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:38.588+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:38.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:38.588+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:38.589 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:38.588+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:38.589 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:38.588+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:38.590 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:38.588+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:39.072 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:39.072+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:39.072 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:39.072+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:39.072 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:39.072+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:39.073 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:39.072+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:39.119 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:39.116+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:39.119 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:39.116+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:39.119 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:39.116+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:39.120 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:39.116+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:39.322 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:39.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:39.323 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:39.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:39.323 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:39.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:39.323 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:39.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:39.607 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:39.604+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:39.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:39.604+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:39.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:39.604+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:39.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:39.604+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:39.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:39.608+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:40.024 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:40.024+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:40.024 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:40.024+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:40.024 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:40.024+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:40.025 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:40.024+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:40.116 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:40.116+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:40.116 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:40.116+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:40.117 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:40.116+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:40.117 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:40.116+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:40.319 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:40.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:40.319 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:40.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:40.319 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:40.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:40.319 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:40.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:40.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:40.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:40.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:40.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:40.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:40.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:40.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:40.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:40.622 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:40.620+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:41.021 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:41.020+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:41.021 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:41.020+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:41.021 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:41.020+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:41.021 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:41.020+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:41.155 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:41.152+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:41.155 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:41.152+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:41.155 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:41.152+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:41.156 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:41.152+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:41.365 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:41.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:41.365 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:41.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:41.365 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:41.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:41.366 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:41.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:41.620 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:41.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:41.620 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:41.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:41.620 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:41.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:41.620 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:41.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:41.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:41.620+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:42.009 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:42.008+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:42.009 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:42.008+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:42.010 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:42.008+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:42.010 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:42.008+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:42.155 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:42.152+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:42.155 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:42.152+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:42.155 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:42.152+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:42.156 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:42.152+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:42.369 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:42.368+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:42.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:42.368+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:42.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:42.368+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:42.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:42.368+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:42.597 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:42.596+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:42.597 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:42.596+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:42.597 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:42.596+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:42.597 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:42.596+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:42.598 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:42.596+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:43.031 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:43.032+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:43.031 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:43.032+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:43.032 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:43.032+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:43.032 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:43.032+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:43.187 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:43.188+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:43.188 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:43.188+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:43.188 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:43.188+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:43.188 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:43.188+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:43.353 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:43.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:43.354 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:43.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:43.354 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:43.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:43.354 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:43.352+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:43.439 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:05:43.436+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3550 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:05:43.553 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:43.552+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:43.553 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:43.552+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:43.553 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:43.552+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:43.554 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:43.552+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:43.554 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:43.552+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:44.016 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:44.016+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:44.016 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:44.016+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:44.016 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:44.016+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:44.017 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:44.016+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:44.177 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:44.176+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:44.178 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:44.176+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:44.178 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:44.176+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:44.178 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:44.176+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:44.318 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:44.316+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:44.318 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:44.316+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:44.318 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:44.316+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:44.319 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:44.316+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:44.538 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:44.536+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:44.538 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:44.536+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:44.538 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:44.536+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:44.539 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:44.536+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:44.539 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:44.536+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:45.001 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:45.000+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:45.001 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:45.000+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:45.002 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:45.000+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:45.002 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:45.000+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:45.200 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:45.200+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:45.200 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:45.200+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:45.200 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:45.200+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:45.200 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:45.200+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:45.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:45.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:45.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:45.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:45.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:45.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:45.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:45.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:45.491 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:45.492+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:45.491 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:45.492+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:45.491 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:45.492+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:45.491 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:45.492+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:45.492 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:45.492+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:46.006 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:46.004+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:46.006 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:46.004+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:46.006 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:46.004+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:46.007 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:46.004+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:46.219 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:46.220+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:46.219 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:46.220+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:46.220 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:46.220+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:46.220 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:46.220+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:46.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:46.372+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:46.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:46.372+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:46.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:46.372+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:46.371 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:46.372+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:46.521 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:46.520+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:46.522 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:46.520+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:46.522 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:46.520+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:46.522 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:46.520+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:46.522 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:46.520+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:47.026 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:47.028+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:47.026 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:47.028+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:47.026 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:47.028+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:47.027 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:47.028+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:47.192 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:47.192+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:47.192 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:47.192+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:47.193 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:47.192+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:47.193 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:47.192+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:47.323 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:47.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:47.323 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:47.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:47.323 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:47.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:47.324 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:47.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:47.472 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:47.472+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:47.473 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:47.472+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:47.473 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:47.472+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:47.473 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:47.472+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:47.473 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:47.472+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:47.994 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:47.992+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:47.994 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:47.992+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:47.994 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:47.992+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:47.994 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:47.992+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:48.195 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:48.196+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:48.196 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:48.196+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:48.196 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:48.196+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:48.196 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:48.196+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:48.279 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:48.276+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:48.279 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:48.276+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:48.280 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:48.276+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:48.280 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:48.276+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:48.440 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:05:48.440+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3585 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:05:48.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:48.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:48.461 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:48.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:48.461 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:48.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:48.461 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:48.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:48.461 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:48.456+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:49.010 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:49.012+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:49.011 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:49.012+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:49.011 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:49.012+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:49.011 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:49.012+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:49.239 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:49.236+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:49.239 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:49.236+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:49.240 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:49.236+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:49.242 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:49.236+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:49.325 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:49.324+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:49.326 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:49.324+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:49.326 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:49.324+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:49.326 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:49.324+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:49.422 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:49.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:49.423 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:49.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:49.423 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:49.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:49.423 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:49.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:49.423 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:49.420+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:50.001 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:50.000+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:50.001 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:50.000+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:50.002 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:50.000+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:50.002 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:50.000+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:50.192 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:50.192+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:50.193 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:50.192+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:50.193 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:50.192+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:50.193 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:50.192+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:50.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:50.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:50.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:50.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:50.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:50.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:50.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:50.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:50.459 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:50.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:50.459 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:50.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:50.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:50.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:50.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:50.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:50.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:50.456+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:51.039 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:51.040+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:51.039 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:51.040+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:51.039 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:51.040+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:51.040 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:51.040+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:51.171 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:51.172+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:51.171 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:51.172+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:51.171 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:51.172+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:51.171 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:51.172+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:51.340 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:51.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:51.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:51.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:51.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:51.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:51.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:51.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:51.471 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:51.472+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:51.471 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:51.472+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:51.472 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:51.472+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:51.472 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:51.472+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:51.472 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:51.472+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:52.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:52.064+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:52.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:52.064+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:52.064 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:52.064+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:52.064 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:52.064+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:52.209 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:52.208+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:52.210 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:52.208+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:52.210 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:52.208+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:52.210 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:52.208+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:52.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:52.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:52.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:52.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:52.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:52.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:52.388 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:52.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:52.470 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:52.471+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:52.470 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:52.471+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:52.470 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:52.471+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:52.471 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:52.471+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:52.471 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:52.471+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:53.092 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:53.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:53.092 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:53.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:53.093 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:53.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:53.093 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:53.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:53.165 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:53.163+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:53.165 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:53.163+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:53.165 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:53.163+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:53.166 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:53.163+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:53.372 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:53.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:53.372 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:53.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:53.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:53.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:53.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:53.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:53.441 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:05:53.439+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3621 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:05:53.504 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:53.499+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:53.504 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:53.499+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:53.505 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:53.499+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:53.505 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:53.499+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:53.505 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:53.499+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:54.048 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:54.047+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:54.048 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:54.047+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:54.049 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:54.047+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:54.049 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:54.047+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:54.140 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:54.139+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:54.141 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:54.139+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:54.141 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:54.139+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:54.141 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:54.139+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:54.351 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:54.347+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:54.351 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:54.347+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:54.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:54.347+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:54.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:54.347+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:54.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:54.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:54.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:54.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:54.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:54.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:54.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:54.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:54.530 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:54.527+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:55.009 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:55.007+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:55.010 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:55.007+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:55.010 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:55.007+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:55.010 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:55.007+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:55.109 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:55.111+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:55.110 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:55.111+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:55.110 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:55.111+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:55.110 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:55.111+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:55.340 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:55.339+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:55.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:55.339+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:55.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:55.339+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:55.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:55.339+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:55.578 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:55.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:55.578 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:55.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:55.579 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:55.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:55.579 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:55.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:55.579 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:55.575+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:56.038 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:56.039+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:56.039 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:56.039+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:56.039 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:56.039+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:56.039 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:56.039+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:56.117 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:56.115+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:56.117 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:56.115+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:56.118 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:56.115+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:56.118 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:56.115+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:56.302 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:56.299+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:56.302 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:56.299+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:56.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:56.299+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:56.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:56.299+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:56.536 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:56.535+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:56.537 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:56.535+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:56.537 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:56.535+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:56.538 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:56.535+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:56.538 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:56.535+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:57.072 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:57.071+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:57.073 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:57.071+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:57.073 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:57.071+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:57.073 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:57.071+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:57.118 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:57.119+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:57.118 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:57.119+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:57.118 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:57.119+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:57.119 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:57.119+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:57.313 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:57.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:57.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:57.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:57.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:57.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:57.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:57.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:57.551 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:57.551+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:57.551 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:57.551+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:57.551 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:57.551+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:57.551 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:57.551+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:57.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:57.551+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:58.090 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:58.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:58.090 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:58.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:58.090 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:58.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:58.091 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:58.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:58.158 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:58.159+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:58.158 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:58.159+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:58.159 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:58.159+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:58.159 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:58.159+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:58.270 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:58.271+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:58.270 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:58.271+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:58.270 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:58.271+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:58.270 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:58.271+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:58.443 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:05:58.443+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3657 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:05:58.593 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:58.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:58.593 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:58.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:58.593 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:58.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:58.593 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:58.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:58.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:58.591+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:05:59.122 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:59.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:59.122 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:59.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:59.122 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:59.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:59.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:05:59.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:05:59.150 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:59.147+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:59.150 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:59.147+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:59.151 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:59.147+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:59.151 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:05:59.147+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:05:59.222 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:59.223+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:59.223 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:59.223+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:59.223 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:59.223+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:59.223 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:05:59.223+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:05:59.614 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:59.615+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:59.614 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:59.615+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:59.614 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:59.615+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:59.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:59.615+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:05:59.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:05:59.615+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:00.155 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:00.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:00.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:00.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:00.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:00.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:00.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:00.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:00.184 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:00.183+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:00.184 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:00.183+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:00.185 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:00.183+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:00.185 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:00.183+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:00.218 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:00.219+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:00.219 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:00.219+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:00.219 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:00.219+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:00.219 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:00.219+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:00.629 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:00.627+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:00.629 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:00.627+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:00.629 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:00.627+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:00.630 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:00.627+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:00.630 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:00.627+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:01.134 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:01.131+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:01.135 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:01.131+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:01.135 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:01.131+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:01.136 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:01.131+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:01.152 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:01.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:01.153 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:01.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:01.153 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:01.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:01.153 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:01.151+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:01.226 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:01.227+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:01.226 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:01.227+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:01.227 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:01.227+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:01.227 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:01.227+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:01.651 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:01.651+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:01.651 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:01.651+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:01.651 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:01.651+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:01.652 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:01.651+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:01.652 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:01.651+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:02.096 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:02.095+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:02.096 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:02.095+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:02.097 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:02.095+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:02.097 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:02.095+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:02.175 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:02.175+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:02.175 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:02.175+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:02.175 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:02.175+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:02.175 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:02.175+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:02.212 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:02.211+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:02.213 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:02.211+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:02.213 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:02.211+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:02.213 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:02.211+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:02.617 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:02.615+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:02.617 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:02.615+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:02.618 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:02.615+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:02.618 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:02.615+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:02.618 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:02.615+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:03.133 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:03.131+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:03.133 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:03.131+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:03.134 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:03.131+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:03.134 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:03.131+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:03.140 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:03.139+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:03.140 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:03.139+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:03.140 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:03.139+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:03.141 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:03.139+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:03.198 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:03.199+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:03.198 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:03.199+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:03.199 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:03.199+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:03.199 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:03.199+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:03.444 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:06:03.443+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3693 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:06:03.601 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:03.599+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:03.601 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:03.599+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:03.602 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:03.599+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:03.602 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:03.599+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:03.602 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:03.599+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:04.137 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:04.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:04.138 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:04.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:04.138 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:04.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:04.138 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:04.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:04.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:04.163+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:04.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:04.163+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:04.167 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:04.163+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:04.167 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:04.163+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:04.205 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:04.203+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:04.206 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:04.203+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:04.206 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:04.203+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:04.206 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:04.203+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:04.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:04.559+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:04.564 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:04.559+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:04.564 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:04.559+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:04.564 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:04.559+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:04.565 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:04.563+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:05.125 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:05.123+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:05.125 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:05.123+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:05.125 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:05.123+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:05.126 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:05.123+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:05.139 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:05.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:05.139 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:05.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:05.139 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:05.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:05.140 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:05.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:05.226 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:05.227+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:05.226 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:05.227+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:05.227 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:05.227+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:05.227 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:05.227+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:05.601 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:05.603+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:05.602 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:05.603+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:05.602 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:05.603+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:05.602 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:05.603+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:05.603 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:05.603+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:06.093 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:06.091+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:06.093 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:06.091+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:06.093 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:06.091+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:06.094 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:06.091+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:06.098 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:06.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:06.098 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:06.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:06.098 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:06.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:06.099 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:06.095+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:06.193 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:06.195+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:06.194 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:06.195+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:06.194 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:06.195+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:06.194 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:06.195+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:06.639 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:06.635+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:06.640 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:06.635+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:06.640 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:06.635+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:06.640 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:06.635+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:06.640 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:06.635+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:07.065 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:07.063+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:07.065 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:07.063+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:07.066 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:07.063+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:07.066 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:07.063+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:07.070 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:07.071+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:07.071 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:07.071+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:07.071 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:07.071+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:07.071 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:07.071+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:07.165 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:07.163+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:07.165 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:07.163+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:07.165 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:07.163+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:07.166 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:07.163+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:07.650 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:07.647+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:07.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:07.647+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:07.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:07.647+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:07.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:07.647+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:07.656 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:07.647+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:08.023 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:08.023+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:08.024 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:08.023+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:08.024 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:08.023+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:08.024 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:08.023+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:08.112 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:08.111+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:08.113 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:08.111+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:08.113 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:08.111+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:08.113 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:08.111+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:08.126 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:08.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:08.127 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:08.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:08.127 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:08.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:08.127 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:08.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:08.445 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:06:08.443+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3728 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:06:08.652 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:08.651+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:08.652 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:08.651+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:08.652 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:08.651+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:08.653 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:08.651+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:08.653 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:08.651+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:09.008 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:09.007+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:09.008 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:09.007+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:09.009 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:09.007+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:09.009 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:09.007+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:09.080 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:09.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:09.081 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:09.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:09.081 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:09.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:09.081 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:09.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:09.165 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:09.163+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:09.165 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:09.163+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:09.165 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:09.163+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:09.166 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:09.163+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:09.686 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:09.687+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:09.686 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:09.687+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:09.686 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:09.687+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:09.687 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:09.687+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:09.687 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:09.687+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:10.014 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:10.011+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:10.014 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:10.011+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:10.015 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:10.011+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:10.015 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:10.011+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:10.042 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:10.039+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:10.043 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:10.039+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:10.043 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:10.039+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:10.043 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:10.039+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:10.130 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:10.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:10.130 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:10.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:10.130 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:10.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:10.131 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:10.127+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:10.697 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:10.699+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:10.698 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:10.699+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:10.698 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:10.699+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:10.698 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:10.699+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:10.698 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:10.699+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:11.013 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:11.011+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:11.014 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:11.011+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:11.014 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:11.011+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:11.014 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:11.011+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:11.051 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:11.051+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:11.052 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:11.051+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:11.052 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:11.051+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:11.052 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:11.051+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:11.099 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:11.095+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:11.100 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:11.095+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:11.100 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:11.095+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:11.100 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:11.095+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:11.662 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:11.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:11.662 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:11.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:11.662 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:11.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:11.662 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:11.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:11.663 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:11.659+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:12.038 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:12.039+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:12.039 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:12.039+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:12.039 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:12.039+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:12.039 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:12.039+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:12.079 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:12.079+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:12.080 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:12.079+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:12.080 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:12.079+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:12.080 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:12.079+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:12.097 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:12.099+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:12.098 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:12.099+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:12.098 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:12.099+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:12.098 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:12.099+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:12.678 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:12.679+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:12.679 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:12.679+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:12.679 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:12.679+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:12.679 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:12.679+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:12.679 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:12.679+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:13.039 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:13.035+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:13.040 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:13.035+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:13.040 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:13.035+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:13.040 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:13.035+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:13.120 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:13.119+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:13.121 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:13.119+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:13.121 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:13.119+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:13.121 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:13.119+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:13.122 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:13.119+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:13.122 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:13.119+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:13.122 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:13.119+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:13.122 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:13.119+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:13.447 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:06:13.443+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3764 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:06:13.713 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:13.711+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:13.713 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:13.711+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:13.713 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:13.711+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:13.714 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:13.711+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:13.714 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:13.711+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:14.086 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:14.083+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:14.086 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:14.083+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:14.087 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:14.083+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:14.087 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:14.083+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:14.088 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:14.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:14.088 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:14.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:14.088 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:14.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:14.089 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:14.087+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:14.115 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:14.111+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:14.115 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:14.111+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:14.116 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:14.111+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:14.116 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:14.111+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:14.681 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:14.683+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:14.681 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:14.683+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:14.682 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:14.683+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:14.682 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:14.683+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:14.682 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:14.683+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:15.077 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:15.079+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:15.077 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:15.079+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:15.078 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:15.079+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:15.078 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:15.079+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:15.091 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:15.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:15.092 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:15.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:15.092 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:15.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:15.092 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:15.091+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:15.106 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:15.107+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:15.107 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:15.107+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:15.107 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:15.107+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:15.108 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:15.107+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:15.657 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:15.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:15.658 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:15.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:15.658 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:15.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:15.658 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:15.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:15.658 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:15.659+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:16.085 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:16.083+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:16.085 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:16.083+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:16.086 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:16.083+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:16.086 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:16.083+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:16.116 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:16.115+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:16.117 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:16.115+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:16.117 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:16.115+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:16.117 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:16.115+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:16.125 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:16.127+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:16.125 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:16.127+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:16.125 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:16.127+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:16.125 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:16.127+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:16.630 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:16.631+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:16.630 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:16.631+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:16.630 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:16.631+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:16.631 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:16.631+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:16.631 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:16.631+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:17.088 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:17.087+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:17.088 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:17.087+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:17.089 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:17.087+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:17.089 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:17.087+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:17.129 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:17.131+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:17.130 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:17.131+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:17.130 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:17.131+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:17.130 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:17.131+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:17.147 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:17.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:17.148 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:17.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:17.148 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:17.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:17.148 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:17.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:17.636 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:17.635+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:17.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:17.635+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:17.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:17.635+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:17.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:17.635+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:17.638 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:17.635+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:18.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:18.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:18.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:18.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:18.132 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:18.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:18.132 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:18.131+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:18.133 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:18.131+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:18.133 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:18.131+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:18.133 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:18.131+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:18.133 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:18.131+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:18.154 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:18.151+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:18.154 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:18.151+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:18.154 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:18.151+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:18.155 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:18.151+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:18.448 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:06:18.447+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3800 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:06:18.680 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:18.679+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:18.681 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:18.679+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:18.681 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:18.679+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:18.681 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:18.679+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:18.681 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:18.679+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:19.118 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:19.115+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:19.119 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:19.115+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:19.119 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:19.115+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:19.119 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:19.115+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:19.124 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:19.123+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:19.124 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:19.123+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:19.125 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:19.123+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:19.125 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:19.123+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:19.140 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:19.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:19.140 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:19.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:19.140 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:19.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:19.140 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:19.135+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:19.720 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:19.719+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:19.721 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:19.719+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:19.721 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:19.719+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:19.721 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:19.719+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:19.721 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:19.719+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:20.098 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:20.099+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:20.099 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:20.099+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:20.099 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:20.099+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:20.099 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:20.099+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:20.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:20.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:20.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:20.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:20.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:20.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:20.145 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:20.143+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:20.163 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:20.159+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:20.164 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:20.159+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:20.164 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:20.159+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:20.164 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:20.159+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:20.708 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:20.706+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:20.709 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:20.706+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:20.709 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:20.706+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:20.709 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:20.706+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:20.709 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:20.706+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:21.116 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:21.114+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:21.117 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:21.114+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:21.117 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:21.114+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:21.117 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:21.114+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:21.141 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:21.142+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:21.141 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:21.142+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:21.142 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:21.142+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:21.142 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:21.142+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:21.153 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:21.154+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:21.154 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:21.154+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:21.154 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:21.154+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:21.155 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:21.154+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:21.707 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:21.706+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:21.708 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:21.706+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:21.708 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:21.706+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:21.708 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:21.706+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:21.708 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:21.706+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:22.112 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:22.110+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:22.113 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:22.110+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:22.113 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:22.110+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:22.113 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:22.110+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:22.136 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:22.130+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:22.136 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:22.130+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:22.136 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:22.130+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:22.136 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:22.130+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:22.178 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:22.178+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:22.179 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:22.178+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:22.179 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:22.178+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:22.179 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:22.178+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:22.748 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:22.746+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:22.749 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:22.746+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:22.749 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:22.746+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:22.749 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:22.746+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:22.749 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:22.746+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:23.104 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:23.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:23.105 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:23.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:23.105 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:23.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:23.105 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:23.102+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:23.148 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:23.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:23.148 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:23.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:23.149 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:23.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:23.149 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:23.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:23.215 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:23.202+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:23.215 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:23.202+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:23.215 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:23.202+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:23.216 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:23.202+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:23.450 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:06:23.446+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3836 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:06:23.705 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:23.702+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:23.705 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:23.702+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:23.705 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:23.702+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:23.706 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:23.702+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:23.706 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:23.702+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:24.073 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:24.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:24.074 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:24.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:24.074 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:24.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:24.074 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:24.074+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:24.133 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:24.134+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:24.133 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:24.134+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:24.133 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:24.134+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:24.133 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:24.134+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:24.210 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:24.210+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:24.211 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:24.210+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:24.211 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:24.210+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:24.211 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:24.210+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:24.715 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:24.714+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:24.716 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:24.714+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:24.716 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:24.714+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:24.716 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:24.714+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:24.717 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:24.714+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:25.029 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:25.026+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:25.029 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:25.026+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:25.029 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:25.026+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:25.030 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:25.026+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:25.168 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:25.162+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:25.168 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:25.162+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:25.169 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:25.162+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:25.169 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:25.162+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:25.199 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:25.194+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:25.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:25.194+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:25.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:25.194+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:25.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:25.194+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:25.702 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:25.702+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:25.702 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:25.702+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:25.702 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:25.702+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:25.702 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:25.702+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:25.703 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:25.702+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:26.051 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:26.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:26.051 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:26.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:26.052 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:26.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:26.052 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:26.046+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:26.149 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:26.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:26.149 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:26.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:26.149 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:26.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:26.150 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:26.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:26.235 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:26.234+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:26.235 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:26.234+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:26.236 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:26.234+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:26.236 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:26.234+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:26.656 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:26.654+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:26.656 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:26.654+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:26.657 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:26.654+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:26.657 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:26.654+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:26.657 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:26.654+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:27.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:27.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:27.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:27.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:27.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:27.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:27.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:27.054+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:27.161 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:27.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:27.162 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:27.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:27.162 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:27.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:27.162 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:27.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:27.218 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:27.214+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:27.218 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:27.214+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:27.218 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:27.214+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:27.218 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:27.214+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:27.661 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:27.662+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:27.662 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:27.662+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:27.662 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:27.662+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:27.662 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:27.662+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:27.662 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:27.662+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:28.062 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:28.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:28.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:28.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:28.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:28.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:28.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:28.062+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:28.161 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:28.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:28.162 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:28.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:28.162 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:28.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:28.162 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:28.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:28.171 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:28.170+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:28.171 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:28.170+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:28.171 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:28.170+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:28.171 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:28.170+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:28.451 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:06:28.450+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3871 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:06:28.648 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:28.646+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:28.648 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:28.646+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:28.648 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:28.646+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:28.648 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:28.646+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:28.649 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:28.646+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:29.031 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:29.030+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:29.032 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:29.030+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:29.032 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:29.030+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:29.032 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:29.030+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:29.168 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:29.162+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:29.168 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:29.162+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:29.168 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:29.162+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:29.168 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:29.162+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:29.204 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:29.202+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:29.204 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:29.202+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:29.204 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:29.202+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:29.205 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:29.202+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:29.648 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:29.646+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:29.649 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:29.646+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:29.649 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:29.646+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:29.649 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:29.646+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:29.649 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:29.646+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:30.035 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:30.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:30.035 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:30.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:30.035 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:30.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:30.035 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:30.034+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:30.163 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:30.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:30.164 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:30.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:30.164 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:30.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:30.164 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:30.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:30.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:30.198+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:30.201 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:30.198+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:30.201 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:30.198+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:30.201 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:30.198+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:30.690 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:30.690+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:30.690 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:30.690+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:30.691 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:30.690+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:30.691 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:30.690+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:30.691 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:30.690+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:31.062 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:31.058+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:31.062 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:31.058+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:31.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:31.058+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:31.063 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:31.058+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:31.161 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:31.158+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:31.161 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:31.158+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:31.161 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:31.158+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:31.161 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:31.158+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:31.189 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:31.186+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:31.189 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:31.186+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:31.190 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:31.186+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:31.190 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:31.186+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:31.702 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:31.702+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:31.702 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:31.702+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:31.702 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:31.702+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:31.703 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:31.702+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:31.703 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:31.702+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:32.108 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:32.106+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:32.108 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:32.106+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:32.108 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:32.106+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:32.108 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:32.106+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:32.160 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:32.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:32.161 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:32.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:32.161 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:32.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:32.161 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:32.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:32.191 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:32.186+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:32.191 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:32.186+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:32.191 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:32.186+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:32.191 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:32.186+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:32.654 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:32.654+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:32.654 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:32.654+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:32.654 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:32.654+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:32.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:32.654+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:32.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:32.654+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:33.086 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:33.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:33.087 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:33.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:33.087 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:33.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:33.087 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:33.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:33.157 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:33.154+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:33.158 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:33.154+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:33.158 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:33.154+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:33.159 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:33.154+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:33.160 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:33.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:33.161 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:33.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:33.161 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:33.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:33.161 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:33.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:33.453 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:06:33.450+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3907 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:06:33.691 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:33.686+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:33.692 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:33.686+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:33.692 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:33.686+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:33.692 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:33.686+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:33.693 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:33.686+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:34.097 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:34.094+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:34.098 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:34.094+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:34.098 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:34.094+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:34.098 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:34.094+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:34.115 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:34.110+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:34.115 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:34.110+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:34.116 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:34.110+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:34.116 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:34.110+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:34.170 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:34.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:34.171 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:34.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:34.171 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:34.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:34.171 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:34.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:34.649 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:34.650+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:34.650 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:34.650+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:34.650 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:34.650+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:34.651 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:34.650+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:34.651 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:34.650+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:35.091 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:35.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:35.091 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:35.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:35.092 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:35.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:35.092 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:35.086+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:35.142 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:35.142+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:35.142 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:35.142+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:35.143 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:35.142+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:35.143 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:35.142+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:35.165 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:35.162+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:35.166 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:35.162+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:35.166 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:35.162+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:35.166 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:35.162+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:35.684 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:35.678+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:35.684 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:35.678+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:35.684 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:35.678+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:35.685 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:35.678+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:35.685 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:35.678+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:36.094 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:36.094+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:36.095 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:36.094+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:36.095 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:36.094+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:36.095 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:36.094+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:36.131 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:36.126+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:36.131 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:36.126+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:36.131 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:36.126+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:36.132 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:36.126+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:36.174 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:36.174+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:36.174 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:36.174+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:36.174 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:36.174+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:36.175 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:36.174+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:36.686 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:36.682+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:36.686 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:36.682+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:36.687 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:36.682+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:36.687 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:36.682+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:36.687 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:36.682+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:37.065 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:37.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:37.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:37.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:37.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:37.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:37.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:37.066+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:37.165 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:37.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:37.166 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:37.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:37.166 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:37.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:37.167 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:37.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:37.175 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:37.174+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:37.176 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:37.174+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:37.176 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:37.174+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:37.177 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:37.174+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:37.636 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:37.638+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:37.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:37.638+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:37.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:37.638+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:37.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:37.638+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:37.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:37.638+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:38.019 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:38.018+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:38.020 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:38.018+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:38.020 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:38.018+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:38.020 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:38.018+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:38.146 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:38.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:38.147 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:38.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:38.147 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:38.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:38.147 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:38.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:38.191 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:38.190+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:38.192 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:38.190+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:38.192 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:38.190+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:38.192 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:38.190+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:38.454 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:06:38.454+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3943 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:06:38.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:38.594+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:38.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:38.594+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:38.596 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:38.594+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:38.596 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:38.594+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:38.596 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:38.594+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:39.001 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:38.998+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:39.001 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:38.998+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:39.002 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:38.998+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:39.002 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:38.998+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:39.172 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:39.170+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:39.172 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:39.170+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:39.173 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:39.170+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:39.173 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:39.170+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:39.242 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:39.242+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:39.242 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:39.242+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:39.242 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:39.242+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:39.243 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:39.242+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:39.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:39.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:39.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:39.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:39.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:39.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:39.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:39.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:39.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:39.586+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:40.002 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:40.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:40.003 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:40.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:40.003 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:40.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:40.003 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:40.002+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:40.208 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:40.206+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:40.208 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:40.206+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:40.208 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:40.206+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:40.208 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:40.206+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:40.276 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:40.274+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:40.276 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:40.274+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:40.277 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:40.274+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:40.277 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:40.274+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:40.584 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:40.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:40.585 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:40.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:40.585 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:40.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:40.585 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:40.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:40.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:40.586+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:40.965 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:40.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:40.966 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:40.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:40.966 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:40.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:40.966 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:40.966+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:41.228 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:41.230+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:41.229 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:41.230+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:41.229 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:41.230+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:41.229 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:41.230+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:41.279 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:41.278+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:41.280 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:41.278+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:41.280 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:41.278+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:41.280 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:41.278+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:41.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:41.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:41.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:41.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:41.589 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:41.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:41.589 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:41.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:41.589 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:41.586+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:41.922 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:41.922+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:41.923 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:41.922+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:41.923 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:41.922+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:41.923 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:41.922+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:42.252 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:42.254+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:42.253 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:42.254+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:42.253 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:42.254+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:42.253 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:42.254+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:42.268 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:42.266+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:42.268 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:42.266+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:42.268 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:42.266+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:42.268 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:42.266+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:42.554 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:42.554+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:42.554 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:42.554+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:42.554 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:42.554+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:42.555 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:42.554+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:42.555 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:42.554+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:42.889 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:42.890+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:42.889 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:42.890+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:42.889 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:42.890+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:42.890 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:42.890+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:43.213 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:43.214+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:43.213 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:43.214+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:43.213 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:43.214+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:43.213 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:43.214+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:43.269 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:43.270+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:43.269 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:43.270+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:43.269 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:43.270+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:43.269 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:43.270+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:43.455 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:06:43.454+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 3979 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:06:43.603 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:43.602+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:43.604 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:43.602+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:43.604 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:43.602+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:43.604 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:43.602+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:43.604 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:43.602+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:43.922 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:43.922+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:43.923 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:43.922+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:43.923 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:43.922+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:43.923 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:43.922+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:44.251 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:44.250+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:44.251 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:44.250+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:44.252 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:44.250+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:44.252 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:44.250+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:44.255 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:44.254+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:44.256 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:44.254+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:44.256 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:44.254+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:44.256 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:44.254+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:44.602 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:44.602+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:44.603 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:44.602+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:44.603 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:44.602+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:44.603 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:44.602+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:44.603 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:44.602+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:44.903 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:44.902+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:44.903 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:44.902+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:44.904 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:44.902+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:44.904 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:44.902+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:45.251 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:45.250+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:45.252 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:45.250+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:45.252 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:45.250+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:45.252 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:45.250+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:45.287 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:45.286+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:45.287 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:45.286+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:45.288 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:45.286+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:45.288 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:45.286+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:45.558 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:45.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:45.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:45.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:45.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:45.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:45.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:45.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:45.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:45.558+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:45.858 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:45.858+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:45.858 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:45.858+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:45.859 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:45.858+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:45.859 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:45.858+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:46.256 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:46.258+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:46.257 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:46.258+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:46.257 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:46.258+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:46.257 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:46.258+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:46.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:46.302+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:46.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:46.302+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:46.303 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:46.302+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:46.304 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:46.302+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:46.557 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:46.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:46.558 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:46.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:46.558 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:46.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:46.558 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:46.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:46.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:46.558+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:46.905 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:46.906+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:46.905 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:46.906+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:46.906 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:46.906+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:46.906 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:46.906+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:47.271 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:47.270+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:47.271 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:47.270+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:47.271 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:47.270+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:47.271 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:47.270+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:47.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:47.350+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:47.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:47.350+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:47.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:47.350+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:47.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:47.350+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:47.514 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:47.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:47.514 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:47.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:47.514 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:47.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:47.515 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:47.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:47.515 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:47.514+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:47.942 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:47.942+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:47.942 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:47.942+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:47.942 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:47.942+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:47.942 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:47.942+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:48.319 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:48.318+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:48.319 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:48.318+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:48.320 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:48.318+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:48.320 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:48.318+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:48.392 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:48.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:48.392 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:48.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:48.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:48.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:48.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:48.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:48.457 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:06:48.458+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4014 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:06:48.562 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:48.562+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:48.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:48.562+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:48.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:48.562+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:48.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:48.562+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:48.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:48.562+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:48.955 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:48.954+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:48.956 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:48.954+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:48.956 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:48.954+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:48.956 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:48.954+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:49.335 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:49.334+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:49.335 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:49.334+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:49.336 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:49.334+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:49.336 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:49.334+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:49.347 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:49.346+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:49.348 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:49.346+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:49.348 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:49.346+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:49.348 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:49.346+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:49.585 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:49.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:49.585 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:49.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:49.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:49.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:49.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:49.586+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:49.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:49.586+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:49.930 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:49.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:49.931 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:49.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:49.931 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:49.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:49.931 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:49.930+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:50.289 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:50.290+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:50.289 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:50.290+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:50.289 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:50.290+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:50.289 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:50.290+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:50.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:50.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:50.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:50.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:50.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:50.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:50.394 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:50.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:50.576 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:50.574+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:50.577 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:50.574+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:50.577 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:50.574+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:50.577 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:50.574+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:50.578 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:50.574+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:50.942 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:50.941+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:50.942 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:50.941+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:50.943 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:50.941+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:50.943 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:50.941+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:51.276 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:51.273+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:51.277 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:51.273+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:51.277 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:51.273+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:51.277 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:51.273+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:51.397 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:51.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:51.397 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:51.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:51.398 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:51.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:51.398 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:51.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:51.531 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:51.529+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:51.532 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:51.529+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:51.532 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:51.529+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:51.533 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:51.529+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:51.533 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:51.529+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:51.969 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:51.969+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:51.970 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:51.969+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:51.970 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:51.969+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:51.970 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:51.969+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:52.234 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:52.233+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:52.234 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:52.233+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:52.235 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:52.233+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:52.235 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:52.233+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:52.366 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:52.365+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:52.366 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:52.365+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:52.366 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:52.365+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:52.366 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:52.365+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:52.550 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:52.549+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:52.550 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:52.549+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:52.551 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:52.549+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:52.551 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:52.549+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:52.551 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:52.549+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:52.995 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:52.993+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:52.995 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:52.993+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:52.995 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:52.993+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:52.995 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:52.993+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:53.250 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:53.249+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:53.251 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:53.249+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:53.251 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:53.249+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:53.251 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:53.249+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:53.364 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:53.365+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:53.365 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:53.365+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:53.365 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:53.365+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:53.365 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:53.365+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:53.459 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:06:53.457+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4050 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:06:53.508 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:53.505+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:53.508 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:53.505+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:53.508 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:53.505+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:53.508 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:53.505+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:53.509 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:53.505+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:53.995 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:53.993+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:53.996 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:53.993+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:53.996 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:53.993+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:53.996 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:53.993+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:54.235 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:54.233+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:54.236 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:54.233+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:54.236 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:54.233+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:54.236 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:54.233+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:54.391 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:54.389+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:54.392 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:54.389+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:54.392 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:54.389+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:54.392 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:54.389+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:54.501 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:54.501+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:54.502 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:54.501+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:54.502 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:54.501+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:54.502 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:54.501+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:54.503 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:54.501+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:55.037 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:55.037+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:55.037 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:55.037+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:55.037 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:55.037+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:55.038 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:55.037+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:55.251 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:55.249+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:55.251 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:55.249+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:55.252 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:55.249+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:55.252 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:55.249+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:55.415 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:55.413+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:55.415 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:55.413+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:55.415 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:55.413+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:55.416 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:55.413+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:55.538 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:55.537+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:55.538 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:55.537+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:55.539 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:55.537+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:55.539 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:55.537+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:55.539 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:55.537+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:56.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:56.057+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:56.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:56.057+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:56.061 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:56.057+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:56.061 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:56.057+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:56.299 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:56.297+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:56.300 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:56.297+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:56.300 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:56.297+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:56.300 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:56.297+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:56.377 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:56.377+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:56.377 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:56.377+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:56.377 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:56.377+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:56.377 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:56.377+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:56.564 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:56.561+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:56.565 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:56.561+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:56.565 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:56.561+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:56.565 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:56.561+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:56.565 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:56.565+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:57.055 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:57.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:57.055 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:57.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:57.055 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:57.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:57.055 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:57.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:57.275 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:57.273+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:57.276 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:57.273+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:57.276 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:57.273+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:57.276 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:57.273+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:57.425 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:57.425+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:57.426 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:57.425+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:57.426 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:57.425+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:57.426 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:57.425+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:57.610 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:57.609+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:57.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:57.609+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:57.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:57.609+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:57.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:57.609+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:57.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:57.609+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:58.035 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:58.033+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:58.035 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:58.033+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:58.035 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:58.033+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:58.035 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:58.033+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:58.252 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:58.253+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:58.252 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:58.253+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:58.253 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:58.253+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:58.253 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:58.253+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:58.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:58.385+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:58.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:58.385+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:58.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:58.385+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:58.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:58.385+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:58.460 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:06:58.457+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4086 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:06:58.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:58.593+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:58.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:58.593+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:58.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:58.593+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:58.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:58.593+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:58.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:58.593+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:06:59.052 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:59.049+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:59.052 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:59.049+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:59.052 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:59.049+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:59.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:06:59.049+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:06:59.291 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:59.289+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:59.292 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:59.289+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:59.292 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:59.289+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:59.292 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:06:59.289+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:06:59.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:59.381+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:59.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:59.381+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:59.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:59.381+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:59.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:06:59.381+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:06:59.636 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:59.637+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:59.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:59.637+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:59.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:59.637+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:59.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:59.637+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:06:59.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:06:59.637+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:00.095 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:00.093+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:00.095 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:00.093+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:00.096 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:00.093+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:00.096 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:00.093+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:00.330 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:00.329+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:00.331 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:00.329+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:00.331 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:00.329+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:00.331 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:00.329+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:00.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:00.361+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:00.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:00.361+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:00.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:00.361+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:00.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:00.361+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:00.619 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:00.617+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:00.619 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:00.617+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:00.620 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:00.617+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:00.620 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:00.617+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:00.620 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:00.617+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:01.056 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:01.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:01.056 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:01.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:01.056 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:01.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:01.057 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:01.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:01.361 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:01.361+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:01.361 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:01.361+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:01.361 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:01.361+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:01.362 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:01.361+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:01.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:01.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:01.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:01.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:01.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:01.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:01.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:01.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:01.582 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:01.581+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:01.583 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:01.581+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:01.583 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:01.581+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:01.583 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:01.581+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:01.583 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:01.581+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:02.054 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:02.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:02.055 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:02.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:02.055 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:02.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:02.055 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:02.053+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:02.353 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:02.353+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:02.354 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:02.353+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:02.354 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:02.353+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:02.354 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:02.353+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:02.398 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:02.397+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:02.398 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:02.397+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:02.399 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:02.397+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:02.399 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:02.397+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:02.572 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:02.573+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:02.572 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:02.573+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:02.573 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:02.573+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:02.573 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:02.573+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:02.573 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:02.573+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:03.079 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:03.077+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:03.080 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:03.077+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:03.080 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:03.077+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:03.080 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:03.077+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:03.375 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:03.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:03.376 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:03.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:03.376 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:03.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:03.376 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:03.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:03.418 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:03.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:03.419 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:03.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:03.419 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:03.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:03.419 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:03.417+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:03.462 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:07:03.461+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4122 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:07:03.540 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:03.537+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:03.540 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:03.537+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:03.541 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:03.537+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:03.541 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:03.537+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:03.541 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:03.537+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:04.099 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:04.097+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:04.099 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:04.097+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:04.100 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:04.097+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:04.100 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:04.097+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:04.349 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:04.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:04.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:04.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:04.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:04.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:04.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:04.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:04.409 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:04.409+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:04.410 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:04.409+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:04.410 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:04.409+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:04.410 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:04.409+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:04.504 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:04.501+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:04.504 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:04.501+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:04.504 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:04.501+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:04.505 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:04.501+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:04.505 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:04.501+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:05.145 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:05.145+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:05.146 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:05.145+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:05.146 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:05.145+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:05.146 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:05.145+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:05.381 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:05.381+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:05.382 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:05.381+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:05.382 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:05.381+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:05.382 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:05.381+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:05.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:05.393+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:05.394 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:05.393+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:05.394 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:05.393+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:05.394 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:05.393+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:05.462 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:05.461+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:05.462 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:05.461+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:05.462 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:05.461+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:05.462 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:05.461+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:05.463 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:05.461+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:06.158 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:06.157+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:06.158 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:06.157+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:06.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:06.157+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:06.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:06.157+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:06.353 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:06.353+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:06.353 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:06.353+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:06.354 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:06.353+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:06.354 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:06.353+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:06.407 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:06.405+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:06.407 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:06.405+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:06.408 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:06.405+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:06.408 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:06.405+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:06.417 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:06.417+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:06.417 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:06.417+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:06.418 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:06.417+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:06.418 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:06.417+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:06.418 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:06.417+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:07.197 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:07.197+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:07.198 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:07.197+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:07.198 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:07.197+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:07.198 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:07.197+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:07.382 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:07.381+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:07.382 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:07.381+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:07.382 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:07.381+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:07.383 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:07.381+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:07.424 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:07.425+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:07.425 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:07.425+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:07.425 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:07.425+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:07.425 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:07.425+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:07.466 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:07.465+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:07.467 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:07.465+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:07.467 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:07.465+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:07.467 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:07.465+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:07.467 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:07.465+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:08.188 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:08.185+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:08.188 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:08.185+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:08.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:08.185+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:08.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:08.185+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:08.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:08.401+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:08.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:08.401+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:08.402 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:08.401+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:08.402 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:08.401+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:08.427 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:08.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:08.427 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:08.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:08.428 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:08.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:08.428 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:08.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:08.459 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:08.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:08.459 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:08.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:08.459 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:08.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:08.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:08.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:08.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:08.457+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:08.463 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:07:08.461+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4157 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:07:09.169 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:09.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:09.170 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:09.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:09.170 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:09.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:09.170 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:09.169+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:09.430 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:09.429+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:09.431 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:09.429+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:09.431 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:09.429+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:09.431 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:09.429+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:09.432 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:09.433+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:09.433 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:09.433+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:09.433 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:09.433+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:09.433 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:09.433+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:09.433 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:09.433+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:09.442 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:09.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:09.442 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:09.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:09.442 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:09.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:09.443 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:09.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:10.151 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:10.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:10.152 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:10.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:10.152 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:10.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:10.152 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:10.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:10.433 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:10.433+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:10.433 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:10.433+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:10.433 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:10.433+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:10.434 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:10.433+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:10.434 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:10.433+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:10.439 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:10.437+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:10.439 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:10.437+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:10.440 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:10.437+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:10.440 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:10.437+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:10.463 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:10.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:10.463 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:10.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:10.463 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:10.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:10.464 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:10.461+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:11.177 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:11.177+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:11.177 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:11.177+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:11.177 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:11.177+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:11.177 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:11.177+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:11.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:11.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:11.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:11.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:11.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:11.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:11.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:11.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:11.427 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:11.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:11.427 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:11.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:11.428 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:11.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:11.428 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:11.425+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:11.433 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:11.433+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:11.434 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:11.433+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:11.434 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:11.433+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:11.434 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:11.433+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:11.434 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:11.433+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:12.154 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:12.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:12.154 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:12.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:12.154 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:12.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:12.155 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:12.149+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:12.385 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:12.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:12.385 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:12.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:12.385 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:12.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:12.386 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:12.385+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:12.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:12.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:12.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:12.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:12.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:12.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:12.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:12.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:12.446 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:12.445+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:12.447 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:12.445+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:12.447 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:12.445+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:12.447 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:12.445+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:12.447 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:12.445+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:13.116 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:13.113+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:13.117 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:13.113+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:13.117 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:13.113+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:13.117 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:13.113+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:13.363 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:13.365+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:13.364 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:13.365+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:13.364 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:13.365+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:13.364 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:13.365+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:13.376 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:13.377+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:13.376 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:13.377+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:13.377 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:13.377+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:13.377 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:13.377+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:13.398 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:13.397+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:13.398 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:13.397+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:13.399 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:13.397+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:13.399 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:13.397+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:13.399 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:13.397+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:13.465 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:07:13.461+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4193 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:07:14.147 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:14.145+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:14.148 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:14.145+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:14.148 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:14.145+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:14.148 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:14.145+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:14.380 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:14.381+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:14.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:14.381+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:14.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:14.381+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:14.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:14.381+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:14.394 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:14.393+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:14.395 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:14.393+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:14.395 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:14.393+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:14.395 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:14.393+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:14.440 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:14.441+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:14.440 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:14.441+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:14.441 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:14.441+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:14.441 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:14.441+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:14.441 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:14.441+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:15.165 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:15.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:15.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:15.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:15.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:15.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:15.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:15.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:15.392 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:15.393+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:15.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:15.393+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:15.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:15.393+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:15.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:15.393+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:15.441 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:15.433+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:15.442 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:15.433+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:15.442 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:15.433+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:15.442 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:15.433+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:15.444 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:15.445+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:15.445 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:15.445+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:15.445 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:15.445+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:15.445 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:15.445+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:15.445 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:15.445+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:16.130 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:16.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:16.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:16.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:16.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:16.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:16.131 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:16.129+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:16.348 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:16.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:16.349 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:16.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:16.349 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:16.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:16.349 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:16.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:16.414 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:16.413+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:16.415 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:16.413+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:16.415 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:16.413+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:16.415 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:16.413+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:16.464 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:16.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:16.464 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:16.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:16.465 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:16.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:16.465 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:16.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:16.465 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:16.457+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:17.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:17.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:17.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:17.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:17.167 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:17.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:17.167 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:17.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:17.376 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:17.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:17.377 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:17.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:17.377 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:17.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:17.377 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:17.373+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:17.441 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:17.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:17.442 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:17.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:17.442 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:17.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:17.442 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:17.441+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:17.445 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:17.445+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:17.446 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:17.445+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:17.446 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:17.445+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:17.446 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:17.445+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:17.446 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:17.445+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:18.138 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:18.137+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:18.139 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:18.137+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:18.139 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:18.137+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:18.139 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:18.137+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:18.330 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:18.329+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:18.330 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:18.329+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:18.331 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:18.329+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:18.331 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:18.329+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:18.455 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:18.453+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:18.455 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:18.453+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:18.455 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:18.453+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:18.456 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:18.453+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:18.456 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:18.453+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:18.467 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:07:18.465+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4228 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:07:18.488 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:18.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:18.488 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:18.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:18.488 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:18.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:18.488 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:18.485+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:19.165 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:19.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:19.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:19.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:19.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:19.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:19.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:19.165+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:19.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:19.377+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:19.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:19.377+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:19.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:19.377+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:19.380 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:19.377+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:19.467 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:19.465+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:19.468 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:19.465+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:19.468 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:19.465+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:19.468 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:19.465+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:19.468 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:19.465+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:19.514 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:19.513+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:19.515 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:19.513+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:19.515 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:19.513+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:19.515 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:19.513+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:20.177 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:20.173+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:20.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:20.173+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:20.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:20.173+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:20.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:20.173+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:20.347 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:20.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:20.347 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:20.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:20.348 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:20.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:20.348 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:20.345+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:20.507 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:20.505+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:20.507 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:20.505+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:20.507 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:20.505+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:20.507 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:20.505+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:20.508 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:20.505+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:20.551 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:20.549+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:20.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:20.549+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:20.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:20.549+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:20.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:20.549+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:21.197 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:21.197+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:21.198 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:21.197+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:21.198 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:21.197+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:21.198 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:21.197+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:21.351 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:21.349+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:21.351 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:21.349+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:21.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:21.349+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:21.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:21.349+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:21.470 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:21.469+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:21.470 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:21.469+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:21.471 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:21.469+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:21.471 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:21.469+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:21.471 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:21.469+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:21.540 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:21.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:21.540 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:21.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:21.541 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:21.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:21.541 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:21.541+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:22.202 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:22.197+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:22.202 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:22.197+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:22.203 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:22.197+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:22.203 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:22.197+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:22.318 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:22.317+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:22.318 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:22.317+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:22.318 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:22.317+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:22.319 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:22.317+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:22.423 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:22.421+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:22.423 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:22.421+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:22.424 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:22.421+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:22.424 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:22.421+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:22.424 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:22.421+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:22.570 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:22.569+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:22.570 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:22.569+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:22.570 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:22.569+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:22.571 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:22.569+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:23.176 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:23.172+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:23.176 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:23.172+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:23.177 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:23.172+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:23.177 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:23.172+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:23.349 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:23.348+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:23.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:23.348+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:23.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:23.348+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:23.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:23.348+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:23.417 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:23.416+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:23.417 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:23.416+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:23.418 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:23.416+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:23.418 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:23.416+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:23.418 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:23.416+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:23.468 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:07:23.464+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4265 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:07:23.592 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:23.592+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:23.593 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:23.592+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:23.593 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:23.592+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:23.593 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:23.592+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:24.185 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:24.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:24.186 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:24.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:24.186 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:24.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:24.186 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:24.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:24.331 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:24.328+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:24.331 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:24.328+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:24.331 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:24.328+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:24.332 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:24.328+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:24.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:24.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:24.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:24.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:24.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:24.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:24.461 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:24.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:24.461 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:24.456+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:24.575 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:24.572+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:24.575 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:24.572+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:24.575 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:24.572+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:24.576 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:24.572+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:25.184 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:25.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:25.185 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:25.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:25.185 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:25.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:25.185 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:25.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:25.368 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:25.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:25.369 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:25.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:25.369 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:25.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:25.369 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:25.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:25.414 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:25.412+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:25.415 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:25.412+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:25.415 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:25.412+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:25.415 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:25.412+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:25.416 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:25.412+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:25.570 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:25.568+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:25.570 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:25.568+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:25.571 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:25.568+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:25.571 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:25.568+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:26.173 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:26.172+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:26.174 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:26.172+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:26.174 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:26.172+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:26.174 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:26.172+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:26.408 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:26.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:26.408 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:26.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:26.409 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:26.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:26.409 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:26.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:26.464 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:26.464+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:26.465 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:26.464+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:26.465 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:26.464+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:26.465 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:26.464+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:26.465 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:26.464+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:26.568 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:26.568+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:26.568 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:26.568+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:26.568 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:26.568+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:26.569 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:26.568+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:27.148 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:27.144+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:27.149 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:27.144+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:27.149 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:27.144+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:27.149 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:27.144+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:27.388 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:27.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:27.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:27.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:27.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:27.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:27.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:27.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:27.449 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:27.448+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:27.449 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:27.448+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:27.450 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:27.448+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:27.450 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:27.448+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:27.450 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:27.448+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:27.609 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:27.608+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:27.609 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:27.608+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:27.610 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:27.608+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:27.610 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:27.608+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:28.158 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:28.156+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:28.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:28.156+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:28.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:28.156+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:28.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:28.156+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:28.414 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:28.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:28.414 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:28.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:28.414 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:28.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:28.414 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:28.412+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:28.457 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:28.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:28.458 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:28.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:28.458 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:28.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:28.458 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:28.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:28.458 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:28.456+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:28.469 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:07:28.468+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4299 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:07:28.628 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:28.624+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:28.629 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:28.624+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:28.629 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:28.624+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:28.629 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:28.624+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:29.171 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:29.168+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:29.171 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:29.168+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:29.171 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:29.168+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:29.172 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:29.168+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:29.378 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:29.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:29.378 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:29.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:29.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:29.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:29.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:29.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:29.424 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:29.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:29.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:29.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:29.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:29.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:29.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:29.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:29.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:29.424+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:29.657 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:29.656+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:29.658 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:29.656+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:29.658 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:29.656+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:29.658 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:29.656+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:30.161 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:30.160+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:30.161 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:30.160+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:30.162 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:30.160+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:30.162 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:30.160+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:30.361 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:30.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:30.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:30.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:30.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:30.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:30.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:30.360+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:30.430 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:30.428+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:30.430 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:30.428+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:30.430 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:30.428+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:30.431 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:30.428+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:30.431 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:30.428+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:30.623 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:30.624+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:30.623 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:30.624+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:30.624 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:30.624+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:30.624 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:30.624+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:31.191 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:31.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:31.191 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:31.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:31.192 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:31.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:31.192 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:31.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:31.380 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:31.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:31.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:31.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:31.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:31.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:31.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:31.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:31.401 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:31.400+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:31.402 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:31.400+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:31.402 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:31.400+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:31.402 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:31.400+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:31.402 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:31.400+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:31.660 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:31.660+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:31.660 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:31.660+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:31.660 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:31.660+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:31.661 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:31.660+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:32.165 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:32.164+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:32.165 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:32.164+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:32.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:32.164+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:32.166 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:32.164+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:32.346 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:32.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:32.347 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:32.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:32.347 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:32.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:32.347 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:32.344+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:32.377 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:32.376+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:32.377 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:32.376+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:32.377 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:32.376+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:32.378 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:32.376+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:32.378 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:32.376+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:32.636 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:32.636+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:32.637 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:32.636+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:32.637 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:32.636+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:32.637 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:32.636+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:33.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:33.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:33.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:33.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:33.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:33.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:33.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:33.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:33.357 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:33.352+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:33.357 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:33.352+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:33.357 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:33.356+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:33.358 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:33.356+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:33.358 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:33.356+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:33.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:33.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:33.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:33.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:33.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:33.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:33.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:33.380+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:33.471 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:07:33.468+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4336 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:07:33.668 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:33.664+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:33.668 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:33.664+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:33.669 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:33.664+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:33.669 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:33.664+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:34.233 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:34.232+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:34.234 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:34.232+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:34.234 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:34.232+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:34.234 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:34.232+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:34.373 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:34.372+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:34.374 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:34.372+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:34.374 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:34.372+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:34.374 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:34.372+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:34.374 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:34.372+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:34.406 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:34.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:34.406 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:34.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:34.406 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:34.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:34.407 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:34.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:34.699 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:34.700+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:34.700 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:34.700+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:34.700 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:34.700+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:34.700 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:34.700+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:35.188 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:35.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:35.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:35.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:35.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:35.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:35.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:35.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:35.374 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:35.372+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:35.375 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:35.372+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:35.375 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:35.372+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:35.375 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:35.372+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:35.375 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:35.372+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:35.402 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:35.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:35.403 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:35.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:35.403 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:35.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:35.403 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:35.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:35.745 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:35.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:35.745 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:35.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:35.745 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:35.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:35.745 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:35.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:36.197 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:36.196+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:36.198 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:36.196+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:36.198 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:36.196+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:36.198 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:36.196+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:36.413 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:36.412+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:36.414 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:36.412+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:36.414 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:36.412+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:36.414 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:36.412+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:36.414 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:36.412+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:36.419 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:36.416+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:36.420 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:36.416+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:36.420 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:36.416+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:36.420 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:36.416+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:36.698 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:36.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:36.698 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:36.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:36.698 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:36.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:36.699 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:36.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:37.152 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:37.152+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:37.152 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:37.152+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:37.153 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:37.152+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:37.153 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:37.152+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:37.376 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:37.376+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:37.376 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:37.376+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:37.376 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:37.376+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:37.377 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:37.376+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:37.377 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:37.376+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:37.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:37.424+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:37.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:37.424+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:37.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:37.424+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:37.428 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:37.424+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:37.734 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:37.732+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:37.735 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:37.732+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:37.735 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:37.732+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:37.735 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:37.732+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:38.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:38.176+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:38.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:38.176+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:38.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:38.176+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:38.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:38.176+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:38.392 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:38.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:38.392 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:38.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:38.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:38.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:38.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:38.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:38.421 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:38.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:38.421 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:38.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:38.422 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:38.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:38.422 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:38.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:38.422 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:38.420+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:38.472 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:07:38.472+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4371 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:07:38.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:38.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:38.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:38.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:38.728 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:38.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:38.728 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:38.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:39.217 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:39.216+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:39.217 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:39.216+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:39.218 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:39.216+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:39.218 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:39.216+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:39.385 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:39.384+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:39.386 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:39.384+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:39.386 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:39.384+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:39.386 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:39.384+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:39.386 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:39.384+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:39.403 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:39.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:39.403 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:39.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:39.403 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:39.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:39.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:39.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:39.742 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:39.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:39.743 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:39.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:39.743 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:39.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:39.743 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:39.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:40.185 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:40.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:40.185 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:40.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:40.185 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:40.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:40.185 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:40.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:40.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:40.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:40.402 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:40.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:40.402 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:40.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:40.402 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:40.400+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:40.421 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:40.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:40.421 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:40.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:40.421 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:40.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:40.422 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:40.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:40.422 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:40.420+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:40.705 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:40.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:40.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:40.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:40.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:40.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:40.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:40.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:41.164 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:41.160+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:41.165 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:41.160+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:41.165 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:41.160+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:41.165 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:41.160+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:41.380 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:41.380+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:41.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:41.380+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:41.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:41.380+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:41.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:41.380+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:41.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:41.380+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:41.403 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:41.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:41.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:41.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:41.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:41.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:41.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:41.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:41.694 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:41.692+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:41.694 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:41.692+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:41.694 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:41.692+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:41.695 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:41.692+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:42.142 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:42.140+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:42.142 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:42.140+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:42.142 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:42.140+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:42.142 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:42.140+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:42.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:42.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:42.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:42.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:42.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:42.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:42.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:42.388+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:42.422 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:42.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:42.423 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:42.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:42.423 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:42.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:42.423 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:42.420+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:42.423 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:42.420+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:42.740 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:42.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:42.740 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:42.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:42.741 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:42.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:42.741 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:42.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:43.139 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:43.140+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:43.140 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:43.140+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:43.140 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:43.140+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:43.140 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:43.140+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:43.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:43.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:43.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:43.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:43.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:43.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:43.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:43.404+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:43.426 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:43.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:43.426 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:43.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:43.427 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:43.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:43.427 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:43.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:43.427 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:43.424+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:43.474 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:07:43.472+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4408 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:07:43.781 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:43.780+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:43.782 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:43.780+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:43.782 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:43.780+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:43.782 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:43.780+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:44.147 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:44.148+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:44.148 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:44.148+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:44.148 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:44.148+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:44.148 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:44.148+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:44.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:44.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:44.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:44.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:44.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:44.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:44.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:44.376+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:44.452 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:44.452+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:44.452 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:44.452+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:44.452 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:44.452+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:44.453 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:44.452+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:44.453 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:44.452+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:44.815 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:44.812+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:44.816 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:44.812+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:44.816 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:44.812+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:44.816 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:44.812+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:45.114 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:45.112+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:45.115 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:45.112+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:45.115 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:45.112+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:45.115 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:45.112+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:45.385 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:45.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:45.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:45.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:45.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:45.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:45.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:45.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:45.427 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:45.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:45.427 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:45.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:45.428 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:45.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:45.428 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:45.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:45.428 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:45.424+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:45.790 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:45.788+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:45.790 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:45.788+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:45.790 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:45.788+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:45.790 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:45.788+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:46.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:46.140+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:46.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:46.140+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:46.144 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:46.140+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:46.145 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:46.140+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:46.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:46.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:46.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:46.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:46.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:46.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:46.390 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:46.384+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:46.402 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:46.400+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:46.402 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:46.400+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:46.403 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:46.400+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:46.403 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:46.400+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:46.403 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:46.400+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:46.772 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:46.772+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:46.772 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:46.772+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:46.773 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:46.772+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:46.773 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:46.772+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:47.120 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:47.120+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:47.120 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:47.120+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:47.121 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:47.120+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:47.121 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:47.120+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:47.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:47.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:47.342 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:47.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:47.342 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:47.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:47.342 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:47.340+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:47.446 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:47.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:47.447 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:47.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:47.447 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:47.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:47.447 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:47.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:47.447 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:47.424+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:47.735 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:47.728+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:47.735 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:47.728+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:47.735 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:47.728+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:47.736 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:47.728+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:48.071 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:48.072+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:48.071 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:48.072+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:48.072 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:48.072+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:48.072 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:48.072+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:48.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:48.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:48.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:48.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:48.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:48.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:48.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:48.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:48.443 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:48.440+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:48.443 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:48.440+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:48.444 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:48.440+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:48.444 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:48.440+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:48.444 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:48.440+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:48.475 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:07:48.472+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4442 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:07:48.741 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:48.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:48.741 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:48.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:48.742 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:48.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:48.742 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:48.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:49.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:49.052+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:49.054 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:49.052+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:49.054 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:49.052+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:49.054 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:49.052+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:49.294 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:49.292+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:49.294 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:49.292+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:49.294 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:49.292+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:49.295 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:49.292+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:49.441 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:49.440+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:49.441 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:49.440+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:49.441 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:49.440+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:49.442 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:49.440+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:49.442 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:49.440+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:49.779 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:49.780+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:49.779 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:49.780+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:49.779 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:49.780+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:49.779 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:49.780+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:50.100 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:50.096+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:50.101 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:50.096+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:50.101 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:50.096+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:50.101 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:50.096+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:50.301 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:50.300+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:50.301 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:50.300+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:50.302 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:50.300+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:50.302 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:50.300+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:50.479 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:50.480+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:50.479 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:50.480+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:50.480 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:50.480+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:50.480 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:50.480+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:50.480 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:50.480+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:50.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:50.788+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:50.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:50.788+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:50.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:50.788+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:50.790 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:50.788+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:51.066 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:51.064+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:51.067 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:51.064+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:51.067 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:51.064+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:51.067 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:51.064+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:51.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:51.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:51.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:51.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:51.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:51.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:51.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:51.336+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:51.470 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:51.468+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:51.471 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:51.468+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:51.471 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:51.468+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:51.471 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:51.468+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:51.471 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:51.468+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:51.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:51.772+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:51.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:51.772+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:51.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:51.772+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:51.775 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:51.772+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:52.077 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:52.076+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:52.078 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:52.076+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:52.078 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:52.076+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:52.078 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:52.076+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:52.312 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:52.308+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:52.312 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:52.308+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:52.312 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:52.308+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:52.312 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:52.308+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:52.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:52.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:52.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:52.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:52.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:52.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:52.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:52.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:52.426 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:52.424+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:52.758 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:52.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:52.758 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:52.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:52.758 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:52.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:52.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:52.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:53.056 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:53.056+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:53.056 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:53.056+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:53.057 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:53.056+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:53.057 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:53.056+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:53.316 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:53.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:53.317 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:53.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:53.317 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:53.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:53.317 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:53.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:53.409 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:53.408+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:53.409 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:53.408+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:53.410 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:53.408+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:53.410 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:53.408+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:53.410 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:53.408+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:53.478 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:07:53.476+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4479 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:07:53.768 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:53.768+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:53.768 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:53.768+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:53.768 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:53.768+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:53.768 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:53.768+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:54.095 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:54.096+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:54.095 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:54.096+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:54.095 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:54.096+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:54.096 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:54.096+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:54.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:54.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:54.316 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:54.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:54.316 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:54.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:54.316 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:54.312+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:54.453 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:54.452+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:54.453 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:54.452+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:54.453 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:54.452+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:54.454 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:54.452+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:54.454 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:54.452+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:54.798 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:54.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:54.798 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:54.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:54.798 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:54.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:54.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:54.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:55.101 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:55.100+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:55.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:55.100+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:55.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:55.100+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:55.102 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:55.100+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:55.365 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:55.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:55.366 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:55.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:55.366 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:55.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:55.367 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:55.364+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:55.453 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:55.452+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:55.453 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:55.452+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:55.453 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:55.452+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:55.454 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:55.452+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:55.454 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:55.452+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:55.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:55.764+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:55.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:55.764+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:55.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:55.764+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:55.768 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:55.764+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:56.067 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:56.064+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:56.068 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:56.064+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:56.068 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:56.064+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:56.068 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:56.064+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:56.323 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:56.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:56.324 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:56.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:56.324 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:56.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:56.324 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:56.320+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:56.424 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:56.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:56.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:56.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:56.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:56.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:56.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:56.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:56.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:56.424+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:56.746 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:56.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:56.746 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:56.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:56.746 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:56.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:56.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:56.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:57.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:57.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:57.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:57.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:57.076 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:57.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:57.077 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:57.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:57.279 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:57.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:57.280 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:57.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:57.280 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:57.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:57.280 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:57.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:57.379 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:57.379+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:57.380 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:57.379+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:57.380 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:57.379+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:57.380 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:57.379+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:57.380 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:57.379+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:57.790 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:57.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:57.790 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:57.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:57.790 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:57.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:57.791 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:57.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:58.052 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:58.047+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:58.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:58.047+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:58.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:58.047+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:58.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:58.047+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:58.320 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:58.319+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:58.321 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:58.319+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:58.321 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:58.319+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:58.321 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:58.319+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:58.345 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:58.343+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:58.345 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:58.343+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:58.346 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:58.343+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:58.346 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:58.343+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:58.346 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:58.343+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:58.479 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:07:58.475+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4514 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:07:58.820 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:58.815+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:58.820 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:58.815+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:58.821 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:58.815+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:58.821 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:58.815+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:59.052 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:59.051+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:59.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:59.051+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:59.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:59.051+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:59.053 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:07:59.051+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:07:59.305 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:59.303+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:59.306 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:59.303+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:59.306 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:59.303+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:59.306 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:07:59.303+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:07:59.315 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:59.311+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:59.315 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:59.311+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:59.315 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:59.311+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:59.316 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:59.311+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:07:59.316 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:07:59.311+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:07:59.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:59.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:59.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:59.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:59.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:59.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:07:59.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:07:59.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:00.077 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:00.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:00.077 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:00.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:00.078 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:00.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:00.078 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:00.075+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:00.277 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:00.275+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:00.277 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:00.275+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:00.278 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:00.275+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:00.278 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:00.275+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:00.337 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:00.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:00.337 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:00.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:00.337 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:00.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:00.337 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:00.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:00.338 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:00.335+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:00.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:00.831+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:00.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:00.831+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:00.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:00.831+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:00.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:00.831+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:01.059 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:01.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:01.059 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:01.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:01.059 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:01.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:01.060 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:01.059+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:01.245 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:01.239+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:01.246 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:01.239+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:01.246 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:01.239+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:01.246 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:01.243+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:01.309 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:01.307+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:01.309 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:01.307+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:01.309 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:01.307+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:01.310 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:01.307+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:01.310 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:01.307+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:01.852 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:01.851+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:01.852 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:01.851+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:01.852 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:01.851+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:01.853 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:01.851+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:02.068 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:02.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:02.069 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:02.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:02.069 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:02.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:02.069 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:02.067+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:02.281 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:02.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:02.281 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:02.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:02.281 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:02.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:02.282 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:02.279+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:02.313 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:02.311+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:02.313 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:02.311+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:02.313 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:02.311+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:02.313 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:02.311+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:02.314 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:02.311+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:02.859 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:02.859+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:02.859 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:02.859+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:02.859 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:02.859+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:02.859 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:02.859+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:03.089 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:03.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:03.089 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:03.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:03.089 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:03.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:03.090 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:03.079+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:03.284 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:03.283+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:03.284 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:03.283+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:03.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:03.283+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:03.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:03.283+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:03.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:03.283+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:03.292 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:03.283+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:03.293 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:03.283+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:03.293 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:03.283+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:03.293 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:03.283+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:03.481 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:08:03.479+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4551 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:08:03.909 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:03.907+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:03.909 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:03.907+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:03.910 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:03.907+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:03.910 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:03.907+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:04.127 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:04.123+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:04.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:04.123+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:04.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:04.123+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:04.128 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:04.123+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:04.263 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:04.263+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:04.263 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:04.263+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:04.264 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:04.263+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:04.264 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:04.263+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:04.264 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:04.263+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:04.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:04.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:04.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:04.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:04.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:04.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:04.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:04.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:04.902 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:04.899+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:04.902 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:04.899+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:04.902 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:04.899+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:04.902 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:04.899+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:05.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:05.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:05.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:05.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:05.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:05.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:05.156 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:05.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:05.252 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:05.247+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:05.253 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:05.247+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:05.253 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:05.247+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:05.253 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:05.247+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:05.253 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:05.247+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:05.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:05.379+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:05.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:05.379+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:05.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:05.379+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:05.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:05.379+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:05.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:05.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:05.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:05.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:05.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:05.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:05.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:05.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:06.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:06.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:06.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:06.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:06.123 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:06.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:06.124 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:06.119+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:06.298 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:06.295+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:06.299 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:06.295+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:06.299 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:06.295+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:06.299 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:06.295+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:06.299 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:06.295+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:06.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:06.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:06.339 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:06.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:06.339 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:06.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:06.339 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:06.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:06.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:06.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:06.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:06.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:06.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:06.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:06.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:06.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:07.136 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:07.135+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:07.137 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:07.135+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:07.137 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:07.135+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:07.137 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:07.135+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:07.348 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:07.343+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:07.348 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:07.343+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:07.349 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:07.343+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:07.349 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:07.343+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:07.349 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:07.343+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:07.361 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:07.359+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:07.361 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:07.359+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:07.361 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:07.359+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:07.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:07.359+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:07.930 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:07.919+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:07.930 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:07.919+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:07.931 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:07.919+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:07.931 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:07.919+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:08.161 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:08.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:08.162 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:08.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:08.162 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:08.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:08.162 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:08.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:08.353 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:08.351+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:08.353 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:08.351+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:08.354 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:08.351+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:08.354 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:08.351+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:08.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:08.379+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:08.382 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:08.379+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:08.382 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:08.379+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:08.382 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:08.379+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:08.382 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:08.379+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:08.483 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:08:08.479+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4585 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:08:08.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:08.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:08.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:08.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:08.926 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:08.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:08.926 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:08.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:09.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:09.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:09.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:09.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:09.159 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:09.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:09.160 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:09.155+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:09.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:09.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:09.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:09.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:09.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:09.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:09.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:09.335+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:09.388 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:09.387+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:09.388 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:09.387+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:09.389 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:09.387+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:09.389 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:09.387+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:09.389 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:09.387+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:09.933 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:09.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:09.934 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:09.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:09.934 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:09.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:09.934 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:09.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:10.110 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:10.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:10.110 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:10.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:10.110 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:10.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:10.110 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:10.107+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:10.361 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:10.359+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:10.362 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:10.359+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:10.362 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:10.359+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:10.362 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:10.359+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:10.362 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:10.359+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:10.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:10.367+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:10.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:10.367+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:10.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:10.367+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:10.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:10.367+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:10.893 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:10.891+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:10.893 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:10.891+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:10.894 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:10.891+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:10.894 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:10.891+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:11.132 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:11.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:11.133 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:11.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:11.133 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:11.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:11.133 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:11.127+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:11.373 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:11.371+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:11.374 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:11.371+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:11.374 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:11.371+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:11.374 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:11.371+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:11.374 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:11.371+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:11.375 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:11.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:11.375 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:11.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:11.375 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:11.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:11.375 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:11.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:11.890 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:11.887+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:11.891 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:11.887+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:11.891 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:11.887+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:11.891 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:11.887+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:12.162 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:12.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:12.163 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:12.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:12.163 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:12.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:12.163 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:12.159+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:12.346 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:12.343+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:12.346 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:12.343+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:12.346 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:12.343+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:12.346 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:12.343+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:12.370 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:12.363+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:12.371 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:12.363+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:12.371 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:12.363+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:12.371 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:12.363+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:12.371 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:12.363+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:12.895 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:12.895+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:12.896 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:12.895+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:12.896 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:12.895+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:12.896 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:12.895+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:13.175 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:13.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:13.175 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:13.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:13.175 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:13.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:13.175 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:13.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:13.342 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:13.339+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:13.342 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:13.339+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:13.342 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:13.339+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:13.343 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:13.339+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:13.343 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:13.339+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:13.388 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:13.387+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:13.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:13.387+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:13.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:13.387+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:13.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:13.387+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:13.484 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:08:13.483+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4622 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:08:13.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:13.875+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:13.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:13.875+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:13.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:13.875+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:13.877 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:13.875+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:14.197 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:14.195+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:14.198 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:14.195+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:14.198 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:14.195+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:14.198 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:14.195+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:14.388 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:14.387+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:14.388 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:14.387+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:14.388 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:14.387+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:14.388 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:14.387+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:14.389 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:14.387+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:14.409 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:14.407+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:14.410 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:14.407+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:14.410 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:14.407+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:14.410 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:14.407+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:14.898 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:14.895+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:14.898 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:14.895+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:14.898 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:14.895+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:14.899 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:14.895+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:15.176 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:15.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:15.176 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:15.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:15.176 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:15.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:15.177 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:15.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:15.411 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:15.407+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:15.411 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:15.407+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:15.412 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:15.407+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:15.412 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:15.407+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:15.412 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:15.407+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:15.438 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:15.435+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:15.438 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:15.435+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:15.438 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:15.435+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:15.439 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:15.435+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:15.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:15.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:15.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:15.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:15.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:15.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:15.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:15.923+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:16.225 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:16.223+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:16.225 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:16.223+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:16.226 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:16.223+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:16.226 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:16.223+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:16.366 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:16.363+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:16.366 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:16.363+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:16.366 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:16.363+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:16.367 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:16.363+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:16.367 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:16.363+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:16.445 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:16.439+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:16.445 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:16.439+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:16.445 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:16.439+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:16.446 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:16.439+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:16.887 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:16.887+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:16.887 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:16.887+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:16.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:16.887+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:16.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:16.887+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:17.270 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:17.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:17.271 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:17.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:17.271 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:17.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:17.271 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:17.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:17.374 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:17.375+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:17.375 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:17.375+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:17.375 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:17.375+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:17.375 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:17.375+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:17.375 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:17.375+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:17.449 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:17.443+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:17.449 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:17.443+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:17.449 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:17.443+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:17.449 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:17.443+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:17.907 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:17.907+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:17.908 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:17.907+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:17.908 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:17.907+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:17.908 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:17.907+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:18.233 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:18.231+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:18.233 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:18.231+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:18.234 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:18.231+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:18.234 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:18.231+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:18.340 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:18.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:18.340 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:18.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:18.340 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:18.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:18.340 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:18.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:18.341 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:18.335+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:18.428 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:18.423+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:18.429 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:18.423+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:18.429 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:18.423+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:18.429 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:18.423+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:18.486 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:08:18.483+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4657 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:08:18.861 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:18.855+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:18.861 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:18.855+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:18.861 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:18.855+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:18.861 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:18.855+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:19.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:19.235+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:19.249 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:19.235+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:19.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:19.235+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:19.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:19.235+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:19.302 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:19.299+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:19.302 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:19.299+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:19.303 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:19.299+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:19.303 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:19.299+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:19.303 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:19.299+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:19.461 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:19.459+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:19.461 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:19.459+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:19.462 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:19.459+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:19.462 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:19.459+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:19.903 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:19.903+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:19.903 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:19.903+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:19.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:19.903+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:19.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:19.903+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:20.205 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:20.203+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:20.205 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:20.203+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:20.206 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:20.203+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:20.206 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:20.203+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:20.335 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:20.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:20.335 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:20.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:20.335 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:20.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:20.335 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:20.335+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:20.336 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:20.335+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:20.418 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:20.415+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:20.418 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:20.415+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:20.418 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:20.415+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:20.418 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:20.415+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:20.946 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:20.943+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:20.947 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:20.943+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:20.947 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:20.943+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:20.947 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:20.943+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:21.249 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:21.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:21.249 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:21.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:21.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:21.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:21.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:21.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:21.322 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:21.323+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:21.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:21.323+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:21.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:21.323+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:21.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:21.323+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:21.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:21.323+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:21.388 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:21.387+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:21.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:21.387+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:21.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:21.387+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:21.389 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:21.387+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:21.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:21.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:21.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:21.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:21.937 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:21.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:21.937 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:21.935+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:22.209 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:22.207+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:22.209 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:22.207+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:22.209 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:22.207+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:22.209 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:22.207+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:22.291 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:22.287+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:22.291 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:22.287+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:22.291 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:22.287+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:22.292 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:22.287+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:22.292 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:22.287+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:22.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:22.403+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:22.406 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:22.403+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:22.406 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:22.403+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:22.406 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:22.403+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:22.962 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:22.959+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:22.962 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:22.959+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:22.962 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:22.959+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:22.963 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:22.959+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:23.224 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:23.223+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:23.224 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:23.223+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:23.224 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:23.223+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:23.225 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:23.223+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:23.296 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:23.295+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:23.297 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:23.295+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:23.297 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:23.295+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:23.297 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:23.295+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:23.297 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:23.295+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:23.366 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:23.367+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:23.367 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:23.367+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:23.367 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:23.367+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:23.367 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:23.367+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:23.487 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:08:23.483+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4694 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:08:23.951 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:23.947+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:23.952 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:23.947+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:23.952 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:23.947+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:23.952 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:23.947+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:24.267 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:24.267+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:24.267 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:24.267+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:24.267 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:24.267+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:24.268 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:24.267+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:24.268 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:24.267+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:24.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:24.267+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:24.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:24.267+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:24.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:24.267+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:24.269 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:24.267+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:24.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:24.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:24.371 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:24.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:24.371 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:24.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:24.371 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:24.371+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:24.922 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:24.919+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:24.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:24.919+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:24.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:24.919+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:24.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:24.919+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:25.281 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:25.279+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:25.281 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:25.279+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:25.281 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:25.279+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:25.281 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:25.279+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:25.299 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:25.299+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:25.300 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:25.299+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:25.300 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:25.299+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:25.300 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:25.299+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:25.300 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:25.299+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:25.333 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:25.331+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:25.334 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:25.331+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:25.334 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:25.331+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:25.334 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:25.331+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:25.884 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:25.883+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:25.885 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:25.883+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:25.885 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:25.883+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:25.885 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:25.883+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:26.251 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:26.251+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:26.251 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:26.251+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:26.251 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:26.251+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:26.251 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:26.251+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:26.252 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:26.251+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:26.271 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:26.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:26.271 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:26.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:26.271 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:26.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:26.271 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:26.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:26.348 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:26.343+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:26.348 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:26.343+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:26.348 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:26.343+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:26.348 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:26.343+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:26.893 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:26.891+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:26.894 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:26.891+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:26.894 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:26.891+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:26.894 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:26.891+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:27.201 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:27.199+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:27.201 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:27.199+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:27.202 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:27.199+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:27.202 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:27.199+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:27.202 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:27.199+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:27.221 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:27.219+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:27.222 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:27.219+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:27.222 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:27.219+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:27.222 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:27.219+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:27.375 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:27.375+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:27.375 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:27.375+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:27.376 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:27.375+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:27.376 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:27.375+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:27.915 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:27.911+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:27.915 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:27.911+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:27.916 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:27.911+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:27.916 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:27.911+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:28.168 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:28.163+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:28.169 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:28.163+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:28.169 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:28.163+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:28.169 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:28.163+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:28.169 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:28.163+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:28.231 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:28.231+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:28.232 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:28.231+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:28.232 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:28.231+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:28.232 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:28.231+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:28.331 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:28.327+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:28.332 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:28.327+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:28.332 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:28.327+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:28.332 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:28.327+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:28.489 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:08:28.487+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4728 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:08:28.958 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:28.955+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:28.959 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:28.955+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:28.959 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:28.955+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:28.959 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:28.955+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:29.175 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:29.175+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:29.175 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:29.175+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:29.175 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:29.175+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:29.175 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:29.175+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:29.176 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:29.175+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:29.273 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:29.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:29.273 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:29.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:29.274 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:29.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:29.274 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:29.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:29.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:29.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:29.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:29.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:29.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:29.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:29.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:29.311+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:29.970 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:29.967+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:29.970 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:29.967+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:29.970 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:29.967+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:29.970 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:29.967+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:30.162 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:30.159+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:30.163 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:30.159+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:30.163 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:30.159+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:30.163 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:30.159+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:30.164 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:30.159+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:30.315 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:30.315+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:30.315 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:30.315+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:30.315 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:30.315+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:30.315 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:30.315+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:30.325 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:30.319+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:30.325 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:30.323+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:30.326 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:30.323+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:30.326 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:30.323+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:30.935 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:30.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:30.935 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:30.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:30.935 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:30.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:30.936 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:30.931+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:31.180 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:31.175+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:31.181 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:31.175+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:31.181 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:31.175+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:31.181 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:31.175+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:31.182 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:31.175+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:31.300 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:31.299+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:31.301 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:31.299+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:31.301 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:31.299+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:31.301 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:31.299+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:31.364 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:31.363+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:31.365 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:31.363+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:31.365 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:31.363+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:31.365 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:31.363+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:31.958 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:31.955+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:31.958 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:31.955+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:31.958 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:31.955+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:31.959 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:31.955+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:32.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:32.171+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:32.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:32.171+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:32.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:32.171+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:32.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:32.171+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:32.175 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:32.171+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:32.297 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:32.295+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:32.298 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:32.295+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:32.298 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:32.295+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:32.298 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:32.295+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:32.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:32.395+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:32.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:32.395+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:32.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:32.395+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:32.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:32.395+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:32.981 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:32.978+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:32.982 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:32.978+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:32.982 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:32.978+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:32.982 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:32.978+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:33.167 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:33.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:33.167 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:33.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:33.167 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:33.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:33.167 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:33.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:33.168 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:33.162+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:33.332 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:33.326+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:33.332 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:33.326+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:33.333 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:33.326+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:33.333 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:33.326+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:33.436 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:33.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:33.437 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:33.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:33.437 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:33.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:33.437 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:33.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:33.490 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:08:33.486+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4765 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:08:33.948 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:33.942+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:33.948 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:33.942+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:33.948 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:33.942+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:33.948 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:33.942+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:34.203 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:34.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:34.203 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:34.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:34.203 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:34.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:34.203 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:34.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:34.204 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:34.198+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:34.324 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:34.322+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:34.324 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:34.322+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:34.324 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:34.322+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:34.324 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:34.322+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:34.483 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:34.478+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:34.484 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:34.478+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:34.484 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:34.478+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:34.484 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:34.478+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:34.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:34.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:34.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:34.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:34.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:34.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:34.905 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:34.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:35.166 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:35.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:35.166 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:35.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:35.166 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:35.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:35.167 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:35.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:35.167 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:35.162+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:35.348 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:35.346+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:35.348 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:35.346+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:35.349 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:35.346+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:35.349 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:35.346+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:35.460 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:35.458+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:35.460 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:35.458+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:35.460 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:35.458+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:35.461 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:35.458+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:35.900 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:35.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:35.901 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:35.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:35.901 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:35.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:35.901 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:35.894+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:36.148 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:36.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:36.149 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:36.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:36.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:36.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:36.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:36.146+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:36.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:36.146+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:36.335 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:36.334+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:36.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:36.334+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:36.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:36.334+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:36.336 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:36.334+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:36.410 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:36.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:36.411 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:36.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:36.411 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:36.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:36.411 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:36.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:36.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:36.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:36.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:36.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:36.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:36.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:36.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:36.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:37.188 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:37.182+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:37.189 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:37.182+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:37.189 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:37.182+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:37.189 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:37.182+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:37.190 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:37.182+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:37.374 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:37.374+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:37.374 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:37.374+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:37.374 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:37.374+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:37.374 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:37.374+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:37.418 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:37.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:37.418 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:37.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:37.418 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:37.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:37.419 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:37.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:37.949 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:37.946+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:37.949 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:37.946+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:37.950 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:37.946+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:37.950 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:37.946+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:38.232 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:38.226+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:38.233 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:38.226+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:38.233 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:38.226+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:38.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:38.226+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:38.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:38.226+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:38.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:38.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:38.414 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:38.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:38.414 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:38.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:38.414 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:38.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:38.463 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:38.458+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:38.463 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:38.458+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:38.464 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:38.458+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:38.464 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:38.458+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:38.495 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:08:38.486+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4800 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:08:38.903 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:38.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:38.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:38.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:38.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:38.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:38.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:38.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:39.233 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:39.230+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:39.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:39.230+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:39.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:39.230+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:39.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:39.230+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:39.235 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:39.230+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:39.384 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:39.382+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:39.384 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:39.382+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:39.384 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:39.382+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:39.384 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:39.382+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:39.469 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:39.466+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:39.469 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:39.466+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:39.470 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:39.466+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:39.470 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:39.466+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:39.906 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:39.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:39.906 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:39.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:39.906 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:39.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:39.907 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:39.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:40.271 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:40.270+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:40.271 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:40.270+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:40.271 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:40.270+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:40.272 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:40.270+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:40.272 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:40.270+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:40.370 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:40.370+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:40.370 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:40.370+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:40.370 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:40.370+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:40.371 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:40.370+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:40.441 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:40.438+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:40.441 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:40.438+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:40.442 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:40.438+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:40.442 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:40.438+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:40.859 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:40.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:40.859 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:40.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:40.860 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:40.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:40.860 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:40.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:41.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:41.282+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:41.286 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:41.282+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:41.286 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:41.282+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:41.286 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:41.282+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:41.286 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:41.282+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:41.378 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:41.378+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:41.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:41.378+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:41.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:41.378+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:41.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:41.378+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:41.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:41.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:41.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:41.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:41.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:41.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:41.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:41.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:41.902 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:41.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:41.902 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:41.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:41.903 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:41.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:41.903 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:41.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:42.305 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:42.302+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:42.306 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:42.302+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:42.306 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:42.302+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:42.306 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:42.302+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:42.306 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:42.302+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:42.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:42.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:42.384 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:42.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:42.384 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:42.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:42.384 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:42.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:42.417 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:42.414+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:42.418 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:42.414+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:42.418 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:42.414+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:42.418 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:42.414+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:42.928 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:42.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:42.928 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:42.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:42.929 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:42.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:42.929 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:42.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:43.298 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:43.298+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:43.299 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:43.298+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:43.299 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:43.298+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:43.299 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:43.298+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:43.299 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:43.298+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:43.377 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:43.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:43.378 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:43.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:43.378 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:43.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:43.378 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:43.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:43.421 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:43.414+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:43.421 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:43.414+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:43.421 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:43.414+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:43.422 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:43.414+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:43.494 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:08:43.490+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4837 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:08:43.969 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:43.966+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:43.969 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:43.966+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:43.970 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:43.966+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:43.970 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:43.966+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:44.321 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:44.318+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:44.322 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:44.318+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:44.322 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:44.318+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:44.322 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:44.318+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:44.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:44.318+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:44.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:44.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:44.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:44.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:44.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:44.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:44.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:44.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:44.420 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:44.414+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:44.420 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:44.414+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:44.420 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:44.414+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:44.420 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:44.414+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:44.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:44.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:44.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:44.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:44.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:44.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:44.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:44.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:45.317 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:45.310+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:45.317 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:45.310+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:45.317 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:45.310+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:45.317 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:45.310+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:45.318 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:45.310+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:45.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:45.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:45.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:45.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:45.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:45.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:45.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:45.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:45.415 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:45.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:45.415 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:45.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:45.416 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:45.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:45.416 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:45.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:45.963 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:45.958+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:45.964 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:45.958+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:45.964 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:45.958+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:45.964 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:45.958+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:46.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:46.282+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:46.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:46.282+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:46.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:46.282+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:46.285 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:46.282+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:46.286 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:46.282+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:46.356 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:46.350+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:46.356 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:46.350+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:46.357 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:46.350+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:46.357 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:46.350+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:46.378 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:46.374+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:46.378 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:46.374+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:46.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:46.374+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:46.379 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:46.374+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:46.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:46.918+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:46.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:46.918+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:46.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:46.918+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:46.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:46.918+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:47.259 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:47.258+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:47.259 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:47.258+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:47.259 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:47.258+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:47.259 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:47.258+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:47.260 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:47.258+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:47.387 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:47.382+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:47.387 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:47.382+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:47.388 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:47.382+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:47.388 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:47.382+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:47.394 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:47.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:47.394 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:47.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:47.394 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:47.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:47.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:47.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:47.956 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:47.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:47.956 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:47.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:47.957 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:47.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:47.957 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:47.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:48.237 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:48.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:48.238 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:48.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:48.238 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:48.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:48.239 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:48.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:48.239 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:48.234+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:48.410 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:48.406+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:48.411 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:48.406+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:48.411 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:48.406+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:48.411 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:48.406+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:48.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:48.406+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:48.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:48.406+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:48.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:48.406+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:48.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:48.406+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:48.495 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:08:48.494+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4871 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:08:48.987 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:48.982+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:48.988 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:48.982+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:48.988 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:48.982+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:48.988 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:48.982+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:49.233 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:49.230+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:49.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:49.230+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:49.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:49.230+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:49.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:49.230+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:49.234 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:49.230+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:49.363 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:49.362+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:49.363 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:49.362+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:49.363 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:49.362+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:49.364 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:49.362+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:49.374 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:49.370+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:49.374 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:49.370+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:49.375 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:49.370+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:49.375 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:49.370+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:49.982 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:49.978+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:49.982 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:49.978+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:49.983 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:49.978+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:49.983 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:49.978+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:50.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:50.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:50.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:50.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:50.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:50.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:50.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:50.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:50.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:50.210+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:50.371 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:50.366+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:50.372 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:50.366+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:50.372 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:50.366+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:50.372 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:50.366+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:50.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:50.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:50.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:50.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:50.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:50.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:50.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:50.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:51.009 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:51.006+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:51.010 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:51.006+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:51.010 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:51.006+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:51.010 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:51.006+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:51.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:51.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:51.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:51.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:51.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:51.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:51.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:51.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:51.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:51.218+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:51.396 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:51.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:51.396 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:51.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:51.397 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:51.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:51.397 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:51.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:51.442 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:51.442+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:51.443 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:51.442+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:51.443 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:51.442+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:51.443 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:51.442+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:52.051 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:52.050+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:52.052 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:52.050+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:52.052 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:52.050+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:52.052 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:52.050+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:52.224 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:52.222+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:52.225 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:52.222+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:52.225 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:52.222+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:52.225 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:52.222+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:52.226 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:52.222+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:52.405 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:52.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:52.406 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:52.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:52.406 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:52.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:52.406 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:52.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:52.414 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:52.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:52.415 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:52.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:52.415 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:52.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:52.415 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:52.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:53.020 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:53.018+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:53.020 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:53.018+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:53.020 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:53.018+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:53.021 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:53.018+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:53.177 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:53.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:53.177 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:53.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:53.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:53.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:53.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:53.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:53.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:53.174+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:53.411 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:53.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:53.411 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:53.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:53.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:53.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:53.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:53.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:53.418 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:53.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:53.419 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:53.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:53.419 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:53.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:53.419 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:53.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:53.497 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:08:53.494+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4908 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:08:53.975 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:53.974+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:53.976 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:53.974+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:53.976 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:53.974+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:53.976 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:53.974+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:54.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:54.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:54.175 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:54.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:54.175 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:54.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:54.175 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:54.174+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:54.175 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:54.174+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:54.403 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:54.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:54.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:54.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:54.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:54.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:54.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:54.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:54.430 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:54.430+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:54.431 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:54.430+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:54.431 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:54.430+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:54.431 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:54.430+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:54.945 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:54.946+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:54.946 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:54.946+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:54.946 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:54.946+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:54.946 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:54.946+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:55.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:55.150+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:55.152 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:55.150+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:55.152 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:55.150+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:55.152 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:55.150+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:55.152 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:55.150+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:55.398 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:55.398+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:55.398 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:55.398+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:55.398 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:55.398+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:55.399 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:55.398+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:55.426 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:55.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:55.426 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:55.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:55.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:55.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:55.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:55.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:55.955 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:55.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:55.955 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:55.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:55.955 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:55.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:55.955 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:55.954+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:56.178 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:56.178+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:56.179 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:56.178+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:56.179 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:56.178+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:56.179 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:56.178+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:56.180 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:56.178+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:56.365 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:56.362+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:56.365 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:56.362+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:56.366 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:56.362+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:56.366 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:56.362+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:56.396 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:56.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:56.397 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:56.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:56.397 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:56.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:56.397 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:56.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:56.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:56.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:56.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:56.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:56.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:56.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:56.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:56.922+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:57.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:57.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:57.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:57.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:57.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:57.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:57.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:57.218+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:57.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:57.218+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:57.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:57.358+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:57.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:57.358+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:57.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:57.358+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:57.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:57.358+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:57.388 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:57.386+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:57.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:57.386+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:57.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:57.386+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:57.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:57.386+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:57.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:57.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:57.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:57.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:57.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:57.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:57.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:57.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:58.238 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:58.238+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:58.238 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:58.238+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:58.239 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:58.238+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:58.239 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:58.238+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:58.239 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:58.238+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:58.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:58.334+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:58.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:58.334+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:58.337 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:58.334+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:58.338 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:58.334+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:58.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:58.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:58.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:58.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:58.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:58.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:58.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:58.410+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:58.498 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:08:58.498+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4943 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:08:58.916 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:58.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:58.917 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:58.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:58.917 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:58.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:58.917 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:58.914+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:59.277 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:59.274+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:59.277 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:59.274+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:59.278 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:59.274+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:59.278 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:59.274+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:08:59.278 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:08:59.274+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:08:59.340 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:59.338+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:59.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:59.338+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:59.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:59.338+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:59.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:08:59.338+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:08:59.443 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:59.442+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:59.443 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:59.442+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:59.443 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:59.442+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:59.443 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:08:59.442+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:08:59.942 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:59.942+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:59.943 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:59.942+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:59.943 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:59.942+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:08:59.943 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:08:59.942+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:00.304 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:00.302+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:00.304 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:00.302+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:00.305 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:00.302+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:00.305 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:00.302+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:00.305 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:00.302+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:00.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:00.362+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:00.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:00.362+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:00.369 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:00.362+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:00.370 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:00.362+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:00.491 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:00.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:00.492 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:00.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:00.492 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:00.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:00.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:00.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:00.982 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:00.978+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:00.982 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:00.978+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:00.982 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:00.978+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:00.983 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:00.978+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:01.326 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:01.326+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:01.327 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:01.326+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:01.327 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:01.326+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:01.327 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:01.326+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:01.327 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:01.326+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:01.390 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:01.386+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:01.391 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:01.386+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:01.391 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:01.386+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:01.391 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:01.386+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:01.531 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:01.530+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:01.531 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:01.530+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:01.532 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:01.530+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:01.532 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:01.530+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:01.981 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:01.982+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:01.982 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:01.982+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:01.982 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:01.982+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:01.982 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:01.982+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:02.332 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:02.330+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:02.333 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:02.330+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:02.333 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:02.330+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:02.333 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:02.330+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:02.333 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:02.330+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:02.349 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:02.346+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:02.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:02.346+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:02.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:02.346+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:02.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:02.346+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:02.482 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:02.482+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:02.483 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:02.482+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:02.483 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:02.482+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:02.483 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:02.482+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:02.938 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:02.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:02.938 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:02.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:02.938 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:02.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:02.938 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:02.938+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:03.335 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:03.334+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:03.335 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:03.334+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:03.335 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:03.334+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:03.336 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:03.334+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:03.336 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:03.334+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:03.368 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:03.366+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:03.368 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:03.366+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:03.368 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:03.366+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:03.369 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:03.366+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:03.492 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:03.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:03.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:03.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:03.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:03.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:03.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:03.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:03.500 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:09:03.498+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 4979 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:09:03.903 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:03.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:03.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:03.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:03.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:03.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:03.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:03.902+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:04.322 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:04.322+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:04.322 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:04.322+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:04.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:04.322+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:04.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:04.322+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:04.323 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:04.322+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:04.412 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:04.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:04.413 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:04.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:04.413 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:04.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:04.413 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:04.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:04.527 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:04.526+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:04.528 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:04.526+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:04.528 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:04.526+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:04.528 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:04.526+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:04.907 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:04.906+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:04.908 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:04.906+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:04.908 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:04.906+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:04.908 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:04.906+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:05.358 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:05.358+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:05.359 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:05.358+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:05.359 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:05.358+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:05.359 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:05.358+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:05.360 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:05.358+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:05.398 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:05.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:05.398 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:05.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:05.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:05.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:05.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:05.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:05.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:05.478+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:05.479 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:05.478+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:05.479 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:05.478+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:05.479 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:05.478+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:05.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:05.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:05.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:05.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:05.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:05.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:05.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:05.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:06.373 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:06.370+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:06.373 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:06.370+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:06.374 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:06.370+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:06.374 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:06.370+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:06.374 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:06.370+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:06.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:06.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:06.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:06.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:06.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:06.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:06.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:06.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:06.513 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:06.510+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:06.513 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:06.510+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:06.513 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:06.510+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:06.514 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:06.510+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:06.837 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:06.834+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:06.837 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:06.834+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:06.837 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:06.834+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:06.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:06.834+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:07.355 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:07.354+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:07.356 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:07.354+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:07.356 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:07.354+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:07.356 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:07.354+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:07.377 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:07.374+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:07.377 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:07.374+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:07.378 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:07.374+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:07.378 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:07.374+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:07.378 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:07.378+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:07.497 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:07.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:07.497 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:07.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:07.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:07.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:07.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:07.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:07.837 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:07.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:07.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:07.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:07.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:07.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:07.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:07.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:08.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:08.314+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:08.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:08.314+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:08.314 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:08.314+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:08.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:08.314+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:08.409 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:08.406+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:08.409 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:08.406+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:08.409 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:08.406+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:08.410 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:08.406+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:08.410 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:08.406+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:08.471 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:08.470+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:08.471 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:08.470+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:08.472 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:08.470+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:08.472 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:08.470+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:08.501 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:09:08.502+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5014 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:09:08.812 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:08.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:08.812 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:08.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:08.812 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:08.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:08.812 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:08.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:09.306 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:09.306+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:09.307 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:09.306+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:09.307 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:09.306+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:09.307 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:09.306+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:09.406 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:09.406+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:09.407 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:09.406+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:09.407 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:09.406+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:09.407 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:09.406+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:09.408 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:09.406+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:09.466 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:09.466+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:09.466 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:09.466+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:09.466 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:09.466+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:09.467 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:09.466+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:09.841 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:09.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:09.842 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:09.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:09.842 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:09.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:09.842 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:09.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:10.300 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:10.298+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:10.301 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:10.298+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:10.301 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:10.298+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:10.301 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:10.298+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:10.404 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:10.402+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:10.404 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:10.402+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:10.404 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:10.402+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:10.405 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:10.402+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:10.405 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:10.402+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:10.474 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:10.474+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:10.474 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:10.474+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:10.474 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:10.474+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:10.475 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:10.474+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:10.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:10.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:10.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:10.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:10.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:10.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:10.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:10.886+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:11.272 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:11.269+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:11.272 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:11.269+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:11.272 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:11.269+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:11.272 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:11.269+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:11.452 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:11.449+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:11.452 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:11.449+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:11.452 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:11.449+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:11.453 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:11.449+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:11.453 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:11.449+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:11.454 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:11.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:11.455 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:11.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:11.455 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:11.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:11.455 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:11.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:11.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:11.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:11.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:11.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:11.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:11.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:11.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:11.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:12.239 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:12.237+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:12.239 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:12.237+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:12.240 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:12.237+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:12.240 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:12.237+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:12.439 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:12.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:12.439 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:12.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:12.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:12.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:12.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:12.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:12.479 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:12.477+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:12.479 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:12.477+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:12.479 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:12.477+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:12.479 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:12.477+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:12.480 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:12.477+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:12.840 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:12.829+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:12.840 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:12.829+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:12.840 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:12.829+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:12.841 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:12.829+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:13.199 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:13.197+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:13.199 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:13.197+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:13.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:13.197+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:13.200 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:13.197+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:13.467 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:13.465+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:13.467 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:13.465+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:13.467 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:13.465+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:13.468 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:13.465+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:13.503 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:09:13.501+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5050 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:09:13.505 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:13.501+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:13.506 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:13.501+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:13.506 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:13.501+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:13.506 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:13.501+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:13.507 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:13.501+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:13.853 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:13.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:13.853 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:13.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:13.854 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:13.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:13.854 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:13.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:14.172 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:14.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:14.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:14.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:14.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:14.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:14.173 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:14.169+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:14.447 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:14.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:14.448 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:14.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:14.448 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:14.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:14.448 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:14.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:14.513 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:14.509+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:14.513 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:14.509+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:14.514 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:14.509+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:14.514 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:14.509+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:14.514 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:14.509+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:14.900 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:14.897+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:14.901 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:14.897+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:14.901 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:14.897+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:14.901 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:14.897+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:15.145 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:15.145+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:15.146 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:15.145+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:15.146 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:15.145+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:15.146 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:15.145+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:15.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:15.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:15.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:15.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:15.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:15.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:15.479 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:15.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:15.540 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:15.537+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:15.540 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:15.537+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:15.540 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:15.537+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:15.540 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:15.537+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:15.541 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:15.537+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:15.898 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:15.897+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:15.898 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:15.897+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:15.898 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:15.897+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:15.899 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:15.897+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:16.119 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:16.117+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:16.120 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:16.117+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:16.120 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:16.117+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:16.120 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:16.117+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:16.525 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:16.525+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:16.526 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:16.525+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:16.526 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:16.525+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:16.526 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:16.525+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:16.561 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:16.557+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:16.561 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:16.557+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:16.561 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:16.557+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:16.562 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:16.557+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:16.562 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:16.557+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:16.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:16.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:16.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:16.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:16.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:16.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:16.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:16.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:17.072 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:17.069+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:17.073 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:17.069+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:17.073 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:17.069+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:17.073 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:17.069+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:17.544 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:17.541+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:17.544 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:17.541+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:17.545 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:17.541+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:17.545 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:17.541+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:17.591 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:17.589+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:17.592 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:17.589+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:17.592 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:17.589+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:17.592 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:17.589+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:17.592 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:17.589+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:17.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:17.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:17.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:17.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:17.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:17.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:17.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:17.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:18.075 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:18.073+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:18.076 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:18.073+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:18.076 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:18.073+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:18.076 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:18.073+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:18.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:18.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:18.499 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:18.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:18.499 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:18.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:18.499 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:18.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:18.505 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:09:18.501+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5086 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:09:18.618 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:18.617+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:18.618 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:18.617+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:18.618 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:18.617+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:18.619 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:18.617+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:18.619 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:18.617+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:18.907 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:18.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:18.907 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:18.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:18.907 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:18.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:18.908 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:18.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:19.095 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:19.093+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:19.096 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:19.093+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:19.096 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:19.093+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:19.097 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:19.093+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:19.488 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:19.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:19.488 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:19.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:19.489 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:19.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:19.489 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:19.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:19.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:19.633+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:19.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:19.633+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:19.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:19.633+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:19.638 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:19.633+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:19.638 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:19.633+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:19.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:19.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:19.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:19.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:19.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:19.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:19.912 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:19.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:20.056 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:20.053+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:20.057 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:20.053+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:20.057 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:20.053+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:20.057 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:20.053+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:20.530 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:20.529+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:20.530 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:20.529+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:20.531 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:20.529+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:20.531 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:20.529+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:20.652 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:20.649+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:20.653 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:20.649+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:20.653 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:20.649+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:20.653 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:20.649+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:20.654 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:20.649+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:20.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:20.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:20.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:20.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:20.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:20.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:20.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:20.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:21.014 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:21.013+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:21.015 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:21.013+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:21.015 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:21.013+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:21.015 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:21.013+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:21.579 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:21.577+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:21.580 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:21.577+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:21.580 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:21.577+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:21.580 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:21.577+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:21.679 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:21.677+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:21.679 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:21.677+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:21.679 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:21.677+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:21.680 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:21.677+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:21.680 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:21.677+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:21.887 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:21.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:21.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:21.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:21.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:21.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:21.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:21.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:21.968 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:21.965+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:21.969 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:21.965+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:21.969 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:21.965+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:21.969 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:21.965+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:22.556 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:22.553+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:22.556 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:22.553+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:22.556 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:22.553+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:22.557 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:22.553+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:22.631 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:22.629+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:22.632 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:22.629+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:22.632 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:22.629+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:22.632 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:22.629+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:22.632 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:22.629+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:22.844 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:22.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:22.845 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:22.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:22.845 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:22.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:22.845 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:22.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:22.947 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:22.945+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:22.947 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:22.945+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:22.947 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:22.945+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:22.948 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:22.945+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:23.506 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:09:23.505+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5122 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:09:23.526 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:23.525+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:23.527 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:23.525+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:23.527 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:23.525+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:23.528 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:23.525+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:23.679 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:23.677+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:23.679 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:23.677+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:23.680 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:23.677+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:23.680 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:23.677+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:23.680 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:23.677+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:23.891 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:23.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:23.891 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:23.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:23.891 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:23.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:23.892 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:23.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:23.913 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:23.913+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:23.914 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:23.913+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:23.914 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:23.913+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:23.914 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:23.913+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:24.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:24.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:24.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:24.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:24.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:24.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:24.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:24.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:24.638 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:24.637+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:24.638 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:24.637+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:24.638 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:24.637+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:24.638 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:24.637+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:24.639 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:24.637+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:24.870 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:24.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:24.870 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:24.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:24.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:24.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:24.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:24.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:24.895 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:24.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:24.895 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:24.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:24.896 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:24.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:24.896 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:24.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:25.492 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:25.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:25.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:25.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:25.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:25.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:25.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:25.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:25.606 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:25.605+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:25.606 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:25.605+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:25.606 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:25.605+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:25.607 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:25.605+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:25.607 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:25.605+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:25.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:25.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:25.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:25.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:25.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:25.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:25.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:25.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:25.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:25.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:25.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:25.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:25.904 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:25.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:25.904 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:25.901+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:26.506 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:26.505+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:26.506 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:26.505+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:26.506 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:26.505+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:26.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:26.505+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:26.568 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:26.565+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:26.569 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:26.565+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:26.569 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:26.565+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:26.569 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:26.565+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:26.570 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:26.565+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:26.892 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:26.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:26.892 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:26.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:26.892 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:26.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:26.893 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:26.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:26.930 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:26.929+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:26.930 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:26.929+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:26.930 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:26.929+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:26.930 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:26.929+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:27.551 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:27.549+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:27.551 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:27.549+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:27.552 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:27.549+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:27.552 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:27.549+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:27.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:27.613+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:27.616 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:27.613+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:27.616 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:27.613+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:27.616 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:27.613+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:27.616 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:27.613+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:27.843 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:27.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:27.843 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:27.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:27.844 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:27.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:27.844 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:27.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:27.881 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:27.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:27.882 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:27.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:27.882 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:27.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:27.882 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:27.881+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:28.508 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:09:28.505+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5157 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:09:28.572 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:28.569+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:28.573 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:28.569+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:28.573 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:28.569+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:28.573 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:28.569+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:28.573 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:28.569+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:28.574 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:28.573+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:28.574 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:28.573+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:28.575 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:28.573+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:28.575 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:28.573+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:28.892 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:28.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:28.893 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:28.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:28.893 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:28.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:28.894 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:28.889+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:28.894 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:28.893+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:28.895 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:28.893+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:28.895 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:28.893+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:28.895 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:28.893+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:29.531 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:29.529+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:29.532 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:29.529+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:29.532 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:29.529+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:29.532 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:29.529+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:29.614 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:29.613+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:29.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:29.613+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:29.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:29.613+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:29.616 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:29.613+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:29.616 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:29.613+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:29.852 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:29.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:29.853 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:29.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:29.853 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:29.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:29.853 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:29.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:29.890 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:29.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:29.891 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:29.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:29.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:29.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:29.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:29.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:30.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:30.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:30.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:30.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:30.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:30.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:30.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:30.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:30.652 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:30.649+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:30.652 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:30.649+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:30.652 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:30.649+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:30.653 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:30.649+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:30.653 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:30.649+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:30.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:30.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:30.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:30.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:30.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:30.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:30.890 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:30.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:30.910 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:30.909+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:30.911 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:30.909+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:30.911 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:30.909+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:30.911 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:30.909+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:31.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:31.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:31.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:31.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:31.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:31.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:31.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:31.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:31.668 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:31.665+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:31.669 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:31.665+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:31.669 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:31.665+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:31.669 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:31.665+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:31.669 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:31.665+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:31.919 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:31.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:31.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:31.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:31.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:31.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:31.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:31.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:31.927 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:31.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:31.927 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:31.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:31.927 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:31.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:31.928 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:31.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:32.451 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:32.449+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:32.452 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:32.449+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:32.452 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:32.449+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:32.452 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:32.449+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:32.627 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:32.625+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:32.627 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:32.625+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:32.627 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:32.625+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:32.628 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:32.625+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:32.628 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:32.625+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:32.930 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:32.929+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:32.930 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:32.929+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:32.930 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:32.929+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:32.931 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:32.929+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:32.957 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:32.953+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:32.958 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:32.953+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:32.958 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:32.953+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:32.958 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:32.953+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:33.457 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:33.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:33.457 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:33.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:33.457 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:33.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:33.458 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:33.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:33.510 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:09:33.509+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5193 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:09:33.605 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:33.605+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:33.605 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:33.605+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:33.606 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:33.605+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:33.606 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:33.605+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:33.606 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:33.605+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:33.936 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:33.933+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:33.936 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:33.933+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:33.937 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:33.933+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:33.937 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:33.933+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:33.952 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:33.949+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:33.952 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:33.949+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:33.952 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:33.949+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:33.952 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:33.949+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:34.480 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:34.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:34.480 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:34.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:34.480 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:34.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:34.481 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:34.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:34.633 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:34.633+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:34.634 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:34.633+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:34.634 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:34.633+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:34.634 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:34.633+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:34.635 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:34.633+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:34.902 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:34.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:34.903 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:34.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:34.903 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:34.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:34.903 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:34.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:34.914 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:34.913+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:34.914 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:34.913+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:34.914 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:34.913+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:34.915 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:34.913+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:35.530 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:35.529+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:35.531 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:35.529+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:35.532 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:35.529+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:35.532 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:35.529+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:35.643 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:35.641+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:35.643 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:35.641+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:35.644 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:35.641+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:35.644 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:35.641+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:35.644 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:35.641+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:35.862 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:35.861+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:35.862 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:35.861+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:35.863 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:35.861+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:35.863 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:35.861+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:35.877 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:35.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:35.877 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:35.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:35.877 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:35.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:35.878 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:35.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:36.486 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:36.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:36.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:36.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:36.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:36.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:36.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:36.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:36.684 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:36.681+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:36.684 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:36.681+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:36.684 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:36.681+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:36.685 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:36.681+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:36.685 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:36.681+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:36.867 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:36.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:36.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:36.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:36.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:36.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:36.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:36.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:36.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:36.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:36.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:36.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:36.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:36.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:36.882 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:36.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:37.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:37.505+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:37.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:37.505+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:37.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:37.505+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:37.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:37.505+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:37.669 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:37.669+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:37.670 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:37.669+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:37.670 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:37.669+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:37.670 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:37.669+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:37.670 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:37.669+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:37.848 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:37.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:37.849 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:37.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:37.849 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:37.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:37.849 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:37.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:37.919 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:37.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:37.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:37.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:37.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:37.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:37.921 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:37.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:38.491 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:38.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:38.492 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:38.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:38.492 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:38.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:38.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:38.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:38.511 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:09:38.509+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5229 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:09:38.644 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:38.641+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:38.645 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:38.641+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:38.645 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:38.641+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:38.645 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:38.641+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:38.646 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:38.641+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:38.802 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:38.801+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:38.802 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:38.801+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:38.803 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:38.801+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:38.804 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:38.801+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:38.932 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:38.929+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:38.932 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:38.929+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:38.932 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:38.929+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:38.932 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:38.929+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:39.495 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:39.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:39.495 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:39.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:39.495 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:39.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:39.495 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:39.493+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:39.692 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:39.689+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:39.692 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:39.689+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:39.692 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:39.689+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:39.693 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:39.689+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:39.693 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:39.689+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:39.817 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:39.817+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:39.818 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:39.817+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:39.818 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:39.817+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:39.818 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:39.817+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:39.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:39.913+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:39.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:39.913+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:39.914 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:39.913+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:39.914 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:39.913+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:40.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:40.501+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:40.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:40.501+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:40.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:40.501+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:40.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:40.501+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:40.694 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:40.693+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:40.694 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:40.693+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:40.694 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:40.693+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:40.695 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:40.693+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:40.695 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:40.693+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:40.861 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:40.857+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:40.861 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:40.857+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:40.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:40.857+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:40.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:40.857+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:40.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:40.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:40.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:40.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:40.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:40.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:40.890 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:40.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:41.457 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:41.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:41.458 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:41.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:41.458 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:41.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:41.458 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:41.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:41.646 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:41.645+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:41.647 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:41.645+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:41.647 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:41.645+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:41.647 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:41.645+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:41.647 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:41.645+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:41.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:41.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:41.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:41.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:41.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:41.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:41.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:41.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:41.863 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:41.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:41.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:41.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:41.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:41.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:41.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:41.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:42.428 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:42.425+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:42.428 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:42.425+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:42.428 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:42.425+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:42.429 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:42.425+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:42.658 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:42.657+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:42.659 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:42.657+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:42.659 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:42.657+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:42.659 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:42.657+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:42.659 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:42.657+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:42.823 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:42.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:42.824 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:42.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:42.824 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:42.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:42.824 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:42.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:42.872 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:42.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:42.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:42.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:42.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:42.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:42.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:42.869+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:43.393 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:43.389+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:43.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:43.389+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:43.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:43.389+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:43.394 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:43.389+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:43.513 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:09:43.509+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5265 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:09:43.704 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:43.705+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:43.705 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:43.705+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:43.705 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:43.705+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:43.705 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:43.705+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:43.706 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:43.705+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:43.842 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:43.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:43.842 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:43.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:43.843 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:43.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:43.843 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:43.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:43.854 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:43.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:43.854 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:43.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:43.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:43.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:43.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:43.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:44.361 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:44.361+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:44.362 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:44.361+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:44.362 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:44.361+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:44.362 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:44.361+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:44.709 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:44.705+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:44.709 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:44.705+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:44.709 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:44.705+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:44.710 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:44.705+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:44.710 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:44.705+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:44.872 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:44.869+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:44.872 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:44.869+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:44.873 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:44.869+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:44.873 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:44.869+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:44.878 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:44.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:44.878 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:44.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:44.878 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:44.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:44.878 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:44.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:45.367 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:45.365+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:45.368 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:45.365+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:45.368 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:45.365+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:45.368 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:45.365+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:45.686 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:45.685+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:45.687 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:45.685+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:45.687 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:45.685+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:45.687 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:45.685+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:45.688 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:45.685+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:45.861 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:45.857+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:45.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:45.857+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:45.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:45.857+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:45.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:45.857+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:45.863 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:45.861+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:45.863 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:45.861+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:45.863 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:45.861+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:45.863 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:45.861+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:46.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:46.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:46.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:46.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:46.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:46.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:46.413 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:46.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:46.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:46.653+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:46.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:46.653+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:46.656 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:46.653+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:46.656 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:46.653+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:46.656 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:46.653+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:46.824 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:46.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:46.824 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:46.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:46.825 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:46.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:46.825 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:46.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:46.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:46.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:46.892 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:46.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:46.893 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:46.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:46.893 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:46.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:47.446 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:47.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:47.447 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:47.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:47.447 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:47.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:47.447 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:47.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:47.660 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:47.657+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:47.660 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:47.657+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:47.660 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:47.657+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:47.660 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:47.657+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:47.661 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:47.657+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:47.788 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:47.789+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:47.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:47.789+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:47.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:47.789+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:47.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:47.789+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:47.919 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:47.917+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:47.920 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:47.917+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:47.920 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:47.917+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:47.920 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:47.917+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:48.411 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:48.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:48.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:48.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:48.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:48.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:48.412 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:48.409+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:48.515 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:09:48.513+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5300 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:09:48.613 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:48.609+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:48.614 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:48.609+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:48.614 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:48.613+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:48.614 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:48.613+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:48.614 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:48.613+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:48.776 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:48.773+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:48.777 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:48.773+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:48.777 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:48.773+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:48.777 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:48.773+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:48.946 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:48.945+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:48.947 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:48.945+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:48.947 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:48.945+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:48.947 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:48.945+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:49.388 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:49.385+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:49.388 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:49.385+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:49.388 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:49.385+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:49.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:49.385+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:49.653 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:49.649+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:49.653 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:49.649+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:49.654 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:49.649+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:49.654 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:49.649+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:49.662 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:49.649+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:49.816 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:49.817+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:49.817 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:49.817+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:49.817 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:49.817+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:49.817 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:49.817+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:49.929 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:49.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:49.929 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:49.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:49.929 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:49.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:49.930 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:49.925+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:50.344 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:50.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:50.344 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:50.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:50.345 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:50.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:50.345 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:50.337+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:50.637 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:50.636+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:50.638 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:50.636+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:50.638 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:50.636+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:50.639 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:50.636+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:50.639 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:50.636+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:50.788 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:50.784+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:50.788 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:50.784+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:50.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:50.784+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:50.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:50.784+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:50.893 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:50.888+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:50.893 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:50.888+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:50.894 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:50.888+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:50.894 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:50.888+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:51.294 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:51.292+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:51.294 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:51.292+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:51.294 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:51.292+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:51.294 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:51.292+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:51.609 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:51.604+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:51.610 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:51.604+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:51.610 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:51.604+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:51.610 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:51.604+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:51.610 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:51.604+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:51.809 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:51.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:51.810 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:51.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:51.810 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:51.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:51.810 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:51.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:51.881 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:51.880+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:51.882 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:51.880+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:51.882 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:51.880+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:51.882 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:51.880+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:52.282 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:52.280+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:52.282 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:52.280+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:52.283 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:52.280+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:52.283 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:52.280+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:52.624 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:52.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:52.625 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:52.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:52.625 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:52.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:52.625 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:52.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:52.625 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:52.620+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:52.810 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:52.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:52.810 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:52.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:52.811 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:52.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:52.811 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:52.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:52.880 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:52.876+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:52.881 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:52.876+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:52.881 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:52.876+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:52.881 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:52.876+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:53.284 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:53.280+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:53.285 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:53.280+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:53.285 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:53.280+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:53.285 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:53.280+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:53.516 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:09:53.512+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5336 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:09:53.579 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:53.576+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:53.580 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:53.576+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:53.580 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:53.576+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:53.580 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:53.576+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:53.580 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:53.576+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:53.788 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:53.784+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:53.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:53.784+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:53.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:53.784+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:53.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:53.784+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:53.836 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:53.832+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:53.836 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:53.832+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:53.836 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:53.832+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:53.837 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:53.832+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:54.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:54.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:54.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:54.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:54.291 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:54.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:54.292 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:54.288+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:54.545 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:54.544+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:54.546 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:54.544+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:54.546 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:54.544+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:54.546 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:54.544+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:54.547 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:54.544+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:54.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:54.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:54.800 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:54.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:54.800 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:54.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:54.800 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:54.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:54.858 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:54.848+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:54.858 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:54.848+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:54.859 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:54.848+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:54.859 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:54.848+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:55.253 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:55.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:55.254 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:55.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:55.254 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:55.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:55.254 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:55.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:55.528 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:55.528+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:55.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:55.528+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:55.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:55.528+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:55.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:55.528+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:55.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:55.528+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:55.839 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:55.836+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:55.839 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:55.836+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:55.840 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:55.836+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:55.840 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:55.836+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:55.870 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:55.868+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:55.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:55.868+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:55.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:55.868+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:55.871 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:55.868+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:56.275 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:56.272+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:56.275 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:56.272+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:56.275 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:56.272+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:56.276 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:56.272+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:56.510 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:56.508+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:56.510 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:56.508+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:56.510 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:56.508+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:56.511 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:56.508+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:56.511 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:56.508+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:56.819 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:56.816+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:56.819 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:56.816+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:56.820 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:56.816+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:56.820 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:56.816+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:56.849 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:56.844+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:56.849 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:56.844+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:56.850 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:56.844+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:56.850 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:56.844+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:57.235 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:57.232+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:57.235 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:57.232+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:57.236 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:57.232+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:57.236 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:57.232+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:57.473 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:57.468+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:57.473 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:57.468+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:57.473 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:57.468+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:57.474 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:57.468+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:57.474 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:57.468+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:57.816 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:57.812+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:57.816 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:57.812+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:57.816 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:57.812+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:57.817 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:57.812+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:57.830 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:57.828+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:57.830 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:57.828+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:57.830 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:57.828+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:57.830 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:57.828+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:58.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:58.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:58.189 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:58.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:58.190 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:58.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:58.190 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:58.188+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:58.464 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:58.460+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:58.464 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:58.460+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:58.464 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:58.460+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:58.465 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:58.460+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:58.465 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:58.460+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:58.518 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:09:58.516+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5372 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:09:58.787 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:58.784+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:58.788 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:58.784+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:58.788 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:58.784+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:58.788 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:58.784+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:58.812 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:58.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:58.813 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:58.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:58.813 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:58.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:58.813 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:58.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:59.213 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:59.208+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:59.213 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:59.208+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:59.213 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:59.208+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:59.214 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:09:59.208+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:09:59.498 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:59.496+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:59.499 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:59.496+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:59.499 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:59.496+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:59.499 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:59.496+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:09:59.499 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:09:59.496+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:09:59.738 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:59.736+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:59.738 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:59.736+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:59.739 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:59.736+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:59.739 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:09:59.736+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:09:59.810 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:59.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:59.811 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:59.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:59.811 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:59.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:09:59.811 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:09:59.808+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:00.261 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:00.256+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:00.261 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:00.256+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:00.261 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:00.256+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:00.261 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:00.256+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:00.459 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:00.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:00.459 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:00.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:00.459 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:00.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:00.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:00.456+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:00.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:00.456+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:00.706 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:00.704+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:00.707 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:00.704+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:00.707 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:00.704+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:00.707 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:00.704+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:00.765 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:00.764+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:00.765 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:00.764+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:00.765 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:00.764+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:00.766 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:00.764+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:01.220 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:01.220+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:01.221 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:01.220+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:01.221 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:01.220+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:01.222 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:01.220+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:01.426 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:01.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:01.427 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:01.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:01.427 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:01.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:01.427 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:01.424+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:01.427 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:01.424+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:01.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:01.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:01.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:01.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:01.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:01.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:01.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:01.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:01.750 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:01.748+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:01.751 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:01.748+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:01.751 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:01.748+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:01.751 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:01.748+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:02.187 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:02.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:02.187 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:02.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:02.187 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:02.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:02.188 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:02.184+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:02.446 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:02.444+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:02.446 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:02.444+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:02.447 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:02.444+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:02.447 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:02.444+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:02.447 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:02.444+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:02.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:02.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:02.748 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:02.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:02.748 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:02.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:02.748 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:02.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:02.767 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:02.764+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:02.767 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:02.764+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:02.768 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:02.764+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:02.768 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:02.764+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:03.237 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:03.236+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:03.237 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:03.236+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:03.238 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:03.236+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:03.238 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:03.236+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:03.422 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:03.416+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:03.422 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:03.416+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:03.423 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:03.416+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:03.423 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:03.416+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:03.423 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:03.416+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:03.520 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:10:03.516+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5408 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:10:03.762 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:03.760+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:03.762 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:03.760+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:03.762 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:03.760+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:03.762 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:03.760+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:03.769 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:03.768+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:03.770 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:03.768+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:03.770 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:03.768+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:03.770 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:03.768+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:04.217 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:04.216+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:04.218 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:04.216+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:04.218 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:04.216+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:04.218 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:04.216+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:04.380 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:04.380+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:04.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:04.380+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:04.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:04.380+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:04.381 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:04.380+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:04.382 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:04.380+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:04.743 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:04.740+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:04.744 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:04.740+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:04.744 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:04.740+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:04.744 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:04.740+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:04.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:04.792+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:04.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:04.792+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:04.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:04.792+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:04.795 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:04.792+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:05.262 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:05.260+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:05.263 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:05.260+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:05.263 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:05.260+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:05.264 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:05.260+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:05.388 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:05.384+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:05.388 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:05.384+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:05.389 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:05.384+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:05.389 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:05.384+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:05.389 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:05.384+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:05.730 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:05.728+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:05.731 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:05.728+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:05.731 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:05.728+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:05.731 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:05.728+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:05.798 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:05.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:05.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:05.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:05.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:05.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:05.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:05.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:06.254 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:06.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:06.254 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:06.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:06.255 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:06.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:06.255 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:06.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:06.405 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:06.404+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:06.405 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:06.404+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:06.406 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:06.404+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:06.406 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:06.404+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:06.406 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:06.404+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:06.719 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:06.716+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:06.719 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:06.716+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:06.720 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:06.716+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:06.720 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:06.716+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:06.756 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:06.752+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:06.757 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:06.752+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:06.757 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:06.752+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:06.757 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:06.752+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:07.213 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:07.212+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:07.214 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:07.212+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:07.214 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:07.212+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:07.214 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:07.212+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:07.413 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:07.408+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:07.413 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:07.408+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:07.414 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:07.408+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:07.414 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:07.408+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:07.414 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:07.408+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:07.718 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:07.716+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:07.718 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:07.716+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:07.719 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:07.716+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:07.719 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:07.716+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:07.755 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:07.752+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:07.756 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:07.752+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:07.756 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:07.752+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:07.756 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:07.752+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:08.253 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:08.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:08.253 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:08.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:08.254 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:08.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:08.254 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:08.252+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:08.369 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:08.368+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:08.369 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:08.368+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:08.369 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:08.368+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:08.370 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:08.368+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:08.370 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:08.368+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:08.521 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:10:08.516+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5443 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:10:08.692 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:08.688+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:08.692 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:08.688+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:08.692 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:08.688+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:08.693 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:08.688+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:08.798 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:08.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:08.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:08.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:08.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:08.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:08.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:08.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:09.225 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:09.220+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:09.225 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:09.220+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:09.226 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:09.220+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:09.226 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:09.220+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:09.366 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:09.364+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:09.367 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:09.364+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:09.367 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:09.364+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:09.367 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:09.364+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:09.367 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:09.364+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:09.502 ERROR:paramiko.transport:Socket exception: No route to host (113) 2020-05-18T22:10:09.505 DEBUG:tasks.ceph:Missed logrotate, host unreachable 2020-05-18T22:10:09.736 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:09.736+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:09.737 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:09.736+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:09.737 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:09.736+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:09.737 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:09.736+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:09.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:09.792+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:09.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:09.792+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:09.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:09.792+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:09.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:09.792+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:10.204 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:10.200+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:10.205 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:10.200+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:10.205 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:10.200+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:10.205 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:10.200+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:10.325 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:10.324+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:10.325 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:10.324+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:10.326 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:10.324+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:10.326 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:10.324+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:10.326 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:10.324+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:10.741 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:10.736+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:10.742 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:10.736+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:10.742 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:10.736+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:10.742 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:10.736+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:10.801 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:10.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:10.801 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:10.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:10.802 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:10.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:10.802 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:10.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:11.201 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:11.200+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:11.202 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:11.200+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:11.202 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:11.200+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:11.202 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:11.200+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:11.326 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:11.320+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:11.326 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:11.320+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:11.326 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:11.320+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:11.327 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:11.320+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:11.327 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:11.320+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:11.762 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:11.760+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:11.762 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:11.760+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:11.762 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:11.760+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:11.762 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:11.760+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:11.776 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:11.772+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:11.776 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:11.772+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:11.776 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:11.772+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:11.776 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:11.772+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:12.176 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:12.172+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:12.176 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:12.172+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:12.177 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:12.172+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:12.177 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:12.172+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:12.287 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:12.284+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:12.287 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:12.284+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:12.287 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:12.284+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:12.288 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:12.284+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:12.288 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:12.284+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:12.758 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:12.756+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:12.759 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:12.756+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:12.759 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:12.756+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:12.759 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:12.756+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:12.779 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:12.776+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:12.780 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:12.776+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:12.780 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:12.776+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:12.780 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:12.776+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:13.204 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:13.204+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:13.204 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:13.204+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:13.204 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:13.204+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:13.204 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:13.204+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:13.269 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2020-05-18T22:10:13.269 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2020-05-18T22:10:13.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2020-05-18T22:10:13.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2020-05-18T22:10:13.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2020-05-18T22:10:13.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2020-05-18T22:10:13.271 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2020-05-18T22:10:13.271 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2020-05-18T22:10:13.279 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:13.276+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:13.280 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:13.276+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:13.280 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:13.276+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:13.280 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:13.276+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:13.280 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:13.276+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:13.523 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:10:13.520+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5479 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:10:13.775 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:13.772+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:13.776 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:13.772+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:13.776 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:13.772+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:13.776 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:13.772+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:13.801 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:13.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:13.801 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:13.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:13.801 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:13.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:13.801 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:13.796+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:14.207 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:14.204+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:14.208 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:14.204+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:14.208 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:14.204+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:14.208 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:14.204+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:14.321 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:14.320+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:14.321 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:14.320+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:14.321 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:14.320+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:14.321 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:14.320+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:14.322 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:14.320+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:14.790 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:14.788+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:14.790 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:14.788+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:14.790 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:14.788+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:14.790 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:14.788+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:14.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:14.832+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:14.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:14.832+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:14.835 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:14.832+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:14.836 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:14.832+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:15.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:15.244+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:15.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:15.244+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:15.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:15.244+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:15.246 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:15.244+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:15.271 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:15.268+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:15.272 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:15.268+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:15.272 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:15.268+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:15.272 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:15.268+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:15.272 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:15.268+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:15.793 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:15.792+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:15.794 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:15.792+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:15.794 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:15.792+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:15.794 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:15.792+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:15.830 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:15.828+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:15.831 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:15.828+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:15.831 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:15.828+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:15.831 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:15.828+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:16.247 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:16.244+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:16.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:16.244+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:16.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:16.244+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:16.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:16.244+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:16.248 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:16.244+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:16.249 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:16.244+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:16.249 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:16.244+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:16.249 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:16.244+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:16.249 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:16.244+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:16.780 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:16.776+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:16.781 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:16.776+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:16.781 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:16.776+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:16.781 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:16.776+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:16.832 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:16.832+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:16.833 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:16.832+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:16.833 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:16.832+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:16.833 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:16.832+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:17.217 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:17.212+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:17.217 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:17.212+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:17.217 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:17.212+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:17.217 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:17.212+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:17.218 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:17.212+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:17.296 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:17.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:17.296 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:17.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:17.297 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:17.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:17.297 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:17.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:17.753 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:17.752+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:17.753 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:17.752+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:17.753 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:17.752+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:17.753 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:17.752+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:17.830 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:17.828+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:17.830 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:17.828+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:17.831 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:17.828+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:17.831 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:17.828+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:18.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:18.168+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:18.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:18.168+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:18.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:18.168+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:18.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:18.168+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:18.173 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:18.168+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:18.297 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:18.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:18.297 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:18.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:18.297 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:18.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:18.298 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:18.296+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:18.525 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:10:18.520+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5514 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:10:18.793 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:18.792+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:18.794 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:18.792+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:18.794 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:18.792+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:18.794 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:18.792+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:18.827 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:18.824+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:18.827 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:18.824+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:18.827 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:18.824+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:18.828 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:18.824+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:18.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~6s 2020-05-18T22:10:18.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~6s 2020-05-18T22:10:18.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~6s 2020-05-18T22:10:18.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~6s 2020-05-18T22:10:18.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~6s 2020-05-18T22:10:18.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~6s 2020-05-18T22:10:18.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~6s 2020-05-18T22:10:18.978 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~6s 2020-05-18T22:10:19.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:19.120+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:19.123 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:19.120+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:19.123 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:19.120+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:19.123 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:19.120+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:19.123 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:19.120+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:19.340 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:19.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:19.340 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:19.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:19.340 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:19.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:19.341 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:19.340+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:19.773 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:19.772+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:19.773 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:19.772+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:19.774 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:19.772+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:19.774 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:19.772+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:19.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:19.792+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:19.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:19.792+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:19.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:19.792+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:19.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:19.792+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:20.149 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:20.144+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:20.149 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:20.144+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:20.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:20.144+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:20.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:20.144+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:20.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:20.144+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:20.318 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:20.316+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:20.319 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:20.316+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:20.319 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:20.316+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:20.319 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:20.316+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:20.758 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:20.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:20.758 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:20.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:20.758 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:20.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:20.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:20.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:20.791 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:20.788+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:20.791 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:20.788+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:20.791 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:20.788+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:20.792 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:20.788+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:21.112 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:21.111+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:21.113 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:21.111+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:21.113 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:21.111+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:21.113 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:21.111+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:21.114 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:21.111+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:21.276 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:21.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:21.276 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:21.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:21.277 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:21.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:21.277 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:21.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:21.716 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:21.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:21.716 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:21.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:21.716 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:21.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:21.716 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:21.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:21.802 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:21.799+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:21.802 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:21.799+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:21.803 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:21.799+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:21.803 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:21.799+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:22.161 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:22.159+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:22.162 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:22.159+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:22.162 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:22.159+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:22.162 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:22.159+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:22.162 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:22.159+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:22.273 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:22.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:22.273 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:22.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:22.274 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:22.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:22.274 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:22.271+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:22.728 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:22.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:22.728 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:22.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:22.729 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:22.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:22.729 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:22.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:22.801 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:22.795+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:22.801 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:22.795+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:22.801 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:22.795+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:22.802 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:22.795+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:23.184 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:23.179+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:23.184 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:23.179+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:23.185 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:23.179+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:23.185 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:23.179+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:23.185 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:23.179+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:23.242 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:23.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:23.243 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:23.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:23.243 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:23.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:23.243 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:23.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:23.527 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:10:23.523+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5550 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:10:23.736 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:23.731+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:23.736 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:23.731+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:23.736 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:23.731+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:23.737 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:23.731+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:23.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:23.795+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:23.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:23.795+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:23.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:23.795+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:23.799 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:23.795+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:24.149 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:24.143+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:24.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:24.143+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:24.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:24.143+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:24.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:24.143+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:24.150 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:24.143+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:24.245 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:24.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:24.246 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:24.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:24.246 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:24.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:24.246 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:24.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:24.683 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~11s 2020-05-18T22:10:24.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~11s 2020-05-18T22:10:24.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~11s 2020-05-18T22:10:24.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~11s 2020-05-18T22:10:24.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~11s 2020-05-18T22:10:24.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~11s 2020-05-18T22:10:24.686 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~11s 2020-05-18T22:10:24.686 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~11s 2020-05-18T22:10:24.726 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:24.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:24.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:24.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:24.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:24.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:24.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:24.723+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:24.836 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:24.835+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:24.836 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:24.835+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:24.836 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:24.835+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:24.837 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:24.835+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:25.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:25.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:25.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:25.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:25.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:25.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:25.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:25.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:25.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:25.103+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:25.211 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:25.207+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:25.211 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:25.207+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:25.211 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:25.207+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:25.212 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:25.207+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:25.751 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:25.747+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:25.751 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:25.747+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:25.751 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:25.747+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:25.751 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:25.747+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:25.852 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:25.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:25.852 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:25.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:25.853 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:25.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:25.853 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:25.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:26.129 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:26.123+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:26.130 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:26.123+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:26.130 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:26.123+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:26.131 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:26.123+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:26.131 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:26.123+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:26.177 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:26.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:26.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:26.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:26.178 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:26.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:26.179 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:26.171+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:26.724 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:26.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:26.724 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:26.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:26.725 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:26.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:26.725 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:26.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:26.863 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:26.863+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:26.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:26.863+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:26.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:26.863+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:26.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:26.863+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:27.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:27.099+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:27.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:27.099+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:27.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:27.099+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:27.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:27.099+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:27.103 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:27.099+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:27.211 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:27.207+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:27.211 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:27.207+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:27.211 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:27.207+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:27.212 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:27.207+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:27.750 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:27.747+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:27.750 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:27.747+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:27.751 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:27.747+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:27.751 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:27.747+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:27.893 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:27.891+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:27.893 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:27.891+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:27.893 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:27.891+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:27.894 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:27.891+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:28.123 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:28.119+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:28.124 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:28.119+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:28.124 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:28.119+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:28.124 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:28.119+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:28.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:28.119+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:28.218 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:28.215+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:28.218 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:28.215+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:28.219 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:28.215+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:28.219 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:28.215+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:28.528 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:10:28.523+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5585 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:10:28.777 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:28.771+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:28.778 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:28.771+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:28.778 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:28.771+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:28.778 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:28.771+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:28.847 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:28.843+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:28.847 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:28.843+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:28.847 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:28.843+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:28.847 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:28.843+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:29.077 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:29.071+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:29.077 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:29.071+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:29.077 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:29.071+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:29.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:29.071+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:29.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:29.071+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:29.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:29.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:29.250 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:29.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:29.251 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:29.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:29.251 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:29.247+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:29.771 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:29.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:29.772 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:29.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:29.772 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:29.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:29.772 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:29.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:29.826 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:29.823+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:29.826 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:29.823+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:29.826 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:29.823+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:29.827 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:29.823+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:30.041 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:30.039+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:30.041 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:30.039+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:30.041 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:30.039+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:30.042 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:30.039+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:30.042 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:30.039+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:30.200 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:30.195+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:30.201 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:30.195+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:30.201 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:30.195+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:30.201 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:30.195+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:30.392 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~17s 2020-05-18T22:10:30.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~17s 2020-05-18T22:10:30.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~17s 2020-05-18T22:10:30.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~17s 2020-05-18T22:10:30.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~17s 2020-05-18T22:10:30.394 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~17s 2020-05-18T22:10:30.394 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~17s 2020-05-18T22:10:30.394 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~17s 2020-05-18T22:10:30.740 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:30.735+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:30.741 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:30.735+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:30.741 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:30.735+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:30.741 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:30.735+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:30.820 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:30.819+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:30.820 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:30.819+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:30.820 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:30.819+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:30.820 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:30.819+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:31.071 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:31.067+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:31.071 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:31.067+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:31.071 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:31.067+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:31.071 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:31.067+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:31.072 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:31.067+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:31.247 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:31.243+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:31.247 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:31.243+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:31.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:31.243+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:31.248 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:31.243+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:31.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:31.763+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:31.768 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:31.763+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:31.768 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:31.763+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:31.768 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:31.763+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:31.854 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:31.851+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:31.854 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:31.851+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:31.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:31.851+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:31.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:31.851+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:32.095 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:32.091+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:32.096 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:32.091+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:32.096 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:32.091+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:32.096 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:32.091+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:32.097 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:32.091+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:32.241 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:32.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:32.242 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:32.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:32.242 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:32.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:32.242 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:32.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:32.783 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:32.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:32.783 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:32.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:32.783 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:32.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:32.784 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:32.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:32.837 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:32.831+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:32.837 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:32.831+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:32.838 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:32.831+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:32.838 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:32.831+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:33.124 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:33.123+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:33.124 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:33.123+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:33.124 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:33.123+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:33.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:33.123+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:33.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:33.123+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:33.246 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:33.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:33.246 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:33.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:33.247 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:33.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:33.247 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:33.239+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:33.530 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:10:33.527+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5621 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:10:33.740 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:33.735+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:33.741 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:33.735+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:33.741 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:33.735+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:33.741 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:33.735+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:33.852 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:33.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:33.852 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:33.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:33.853 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:33.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:33.853 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:33.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:34.091 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:34.087+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:34.091 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:34.087+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:34.092 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:34.087+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:34.092 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:34.087+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:34.092 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:34.087+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:34.284 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:34.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:34.284 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:34.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:34.285 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:34.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:34.285 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:34.283+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:34.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:34.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:34.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:34.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:34.790 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:34.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:34.790 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:34.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:34.872 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:34.867+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:34.872 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:34.867+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:34.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:34.867+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:34.873 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:34.867+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:35.087 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:35.083+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:35.087 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:35.083+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:35.088 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:35.083+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:35.088 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:35.083+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:35.088 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:35.083+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:35.318 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:35.315+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:35.319 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:35.315+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:35.319 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:35.315+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:35.319 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:35.315+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:35.801 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:35.795+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:35.801 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:35.795+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:35.802 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:35.795+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:35.802 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:35.795+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:35.895 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:35.891+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:35.895 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:35.891+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:35.896 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:35.891+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:35.896 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:35.891+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:36.100 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~23s 2020-05-18T22:10:36.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~23s 2020-05-18T22:10:36.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~23s 2020-05-18T22:10:36.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~23s 2020-05-18T22:10:36.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~23s 2020-05-18T22:10:36.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~23s 2020-05-18T22:10:36.102 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~23s 2020-05-18T22:10:36.102 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~23s 2020-05-18T22:10:36.128 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:36.123+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:36.129 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:36.123+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:36.129 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:36.123+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:36.129 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:36.123+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:36.129 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:36.123+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:36.362 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:36.359+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:36.362 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:36.359+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:36.363 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:36.359+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:36.363 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:36.359+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:36.850 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:36.847+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:36.851 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:36.847+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:36.851 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:36.847+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:36.851 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:36.847+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:36.893 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:36.887+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:36.893 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:36.887+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:36.894 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:36.887+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:36.894 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:36.887+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:37.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:37.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:37.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:37.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:37.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:37.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:37.106 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:37.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:37.107 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:37.103+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:37.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:37.371+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:37.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:37.371+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:37.372 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:37.371+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:37.373 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:37.371+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:37.845 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:37.839+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:37.846 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:37.839+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:37.846 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:37.839+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:37.846 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:37.839+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:37.866 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:37.863+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:37.866 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:37.863+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:37.866 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:37.863+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:37.866 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:37.863+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:38.103 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:38.099+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:38.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:38.099+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:38.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:38.099+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:38.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:38.099+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:38.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:38.099+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:38.404 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:38.399+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:38.405 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:38.399+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:38.405 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:38.399+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:38.405 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:38.399+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:38.532 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:10:38.527+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5657 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:10:38.803 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:38.799+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:38.803 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:38.799+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:38.803 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:38.799+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:38.804 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:38.799+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:38.823 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:38.819+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:38.823 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:38.819+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:38.823 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:38.819+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:38.823 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:38.819+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:39.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:39.063+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:39.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:39.063+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:39.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:39.063+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:39.068 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:39.063+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:39.068 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:39.063+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:39.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:39.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:39.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:39.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:39.398 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:39.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:39.398 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:39.391+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:39.506 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:10:39.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:10:39.848 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:39.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:39.848 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:39.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:39.849 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:39.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:39.849 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:39.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:39.860 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:39.855+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:39.861 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:39.855+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:39.861 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:39.855+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:39.861 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:39.855+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:40.081 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:40.075+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:40.082 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:40.075+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:40.082 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:40.075+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:40.082 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:40.075+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:40.083 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:40.075+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:40.349 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:40.343+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:40.350 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:40.343+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:40.350 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:40.343+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:40.350 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:40.343+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:40.812 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:40.807+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:40.813 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:40.807+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:40.813 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:40.807+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:40.813 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:40.807+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:40.829 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:40.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:40.829 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:40.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:40.829 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:40.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:40.830 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:40.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:41.110 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:41.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:41.111 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:41.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:41.111 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:41.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:41.111 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:41.103+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:41.111 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:41.107+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:41.377 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:41.375+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:41.377 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:41.375+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:41.378 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:41.375+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:41.378 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:41.375+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:41.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~29s 2020-05-18T22:10:41.809 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~29s 2020-05-18T22:10:41.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~29s 2020-05-18T22:10:41.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~29s 2020-05-18T22:10:41.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~29s 2020-05-18T22:10:41.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~29s 2020-05-18T22:10:41.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~29s 2020-05-18T22:10:41.811 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~29s 2020-05-18T22:10:41.820 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:41.819+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:41.820 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:41.819+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:41.820 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:41.819+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:41.820 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:41.819+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:41.834 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:41.831+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:41.834 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:41.831+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:41.834 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:41.831+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:41.834 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:41.831+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:42.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:42.119+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:42.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:42.119+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:42.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:42.119+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:42.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:42.119+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:42.122 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:42.119+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:42.342 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:42.339+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:42.343 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:42.339+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:42.343 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:42.339+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:42.343 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:42.339+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:42.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:10:42.559 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:10:42.824 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:42.819+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:42.824 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:42.819+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:42.824 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:42.819+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:42.825 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:42.819+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:42.826 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:42.819+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:42.826 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:42.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:42.826 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:42.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:42.826 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:42.823+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:43.086 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:43.079+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:43.086 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:43.079+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:43.086 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:43.079+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:43.086 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:43.079+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:43.087 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:43.079+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:43.326 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:43.323+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:43.326 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:43.323+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:43.326 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:43.323+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:43.327 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:43.323+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:43.533 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:10:43.531+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5693 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:10:43.783 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:43.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:43.784 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:43.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:43.784 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:43.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:43.784 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:43.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:43.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:43.855+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:43.856 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:43.855+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:43.856 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:43.855+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:43.856 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:43.855+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:44.066 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:44.063+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:44.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:44.063+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:44.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:44.063+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:44.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:44.063+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:44.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:44.063+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:44.366 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:44.359+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:44.366 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:44.359+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:44.367 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:44.359+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:44.367 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:44.359+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:44.798 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:44.795+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:44.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:44.795+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:44.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:44.795+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:44.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:44.795+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:44.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:44.899+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:44.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:44.899+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:44.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:44.899+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:44.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:44.899+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:45.076 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:45.071+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:45.077 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:45.071+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:45.077 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:45.071+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:45.077 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:45.071+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:45.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:45.071+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:45.343 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:45.339+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:45.344 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:45.339+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:45.344 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:45.339+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:45.344 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:45.339+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:45.795 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:45.791+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:45.795 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:45.791+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:45.796 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:45.791+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:45.796 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:45.791+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:45.898 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:45.895+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:45.899 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:45.895+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:45.899 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:45.895+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:45.899 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:45.895+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:46.051 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:46.047+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:46.052 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:46.047+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:46.052 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:46.047+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:46.052 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:46.047+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:46.052 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:46.047+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:46.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:46.387+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:46.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:46.387+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:46.390 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:46.387+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:46.390 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:46.387+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:46.760 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:46.759+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:46.761 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:46.759+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:46.761 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:46.759+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:46.761 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:46.759+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:46.874 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:46.871+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:46.875 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:46.871+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:46.875 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:46.871+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:46.875 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:46.871+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:47.049 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:47.047+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:47.049 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:47.047+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:47.049 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:47.047+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:47.050 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:47.047+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:47.050 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:47.047+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:47.384 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:47.383+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:47.385 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:47.383+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:47.385 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:47.383+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:47.385 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:47.383+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:47.518 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~34s 2020-05-18T22:10:47.518 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~34s 2020-05-18T22:10:47.518 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~34s 2020-05-18T22:10:47.519 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~34s 2020-05-18T22:10:47.519 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~34s 2020-05-18T22:10:47.519 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~34s 2020-05-18T22:10:47.519 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~34s 2020-05-18T22:10:47.519 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~34s 2020-05-18T22:10:47.786 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:47.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:47.786 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:47.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:47.787 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:47.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:47.787 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:47.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:47.845 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:47.843+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:47.846 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:47.843+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:47.846 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:47.843+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:47.846 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:47.843+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:48.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:48.015+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:48.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:48.015+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:48.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:48.015+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:48.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:48.015+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:48.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:48.015+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:48.415 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:48.411+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:48.416 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:48.411+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:48.416 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:48.411+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:48.416 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:48.411+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:48.535 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:10:48.531+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5727 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:10:48.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:48.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:48.748 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:48.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:48.748 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:48.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:48.748 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:48.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:48.857 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:48.855+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:48.858 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:48.855+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:48.858 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:48.855+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:48.858 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:48.855+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:48.992 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:48.987+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:48.993 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:48.987+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:48.993 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:48.987+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:48.993 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:48.987+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:48.994 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:48.987+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:49.445 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:49.439+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:49.446 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:49.439+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:49.446 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:49.439+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:49.446 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:49.439+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:49.766 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:49.763+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:49.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:49.763+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:49.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:49.763+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:49.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:49.763+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:49.854 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:49.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:49.854 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:49.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:49.854 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:49.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:49.855 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:49.847+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:49.970 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:49.963+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:49.970 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:49.963+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:49.971 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:49.963+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:49.971 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:49.963+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:49.971 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:49.967+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:50.447 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:50.447+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:50.448 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:50.447+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:50.448 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:50.447+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:50.448 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:50.447+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:50.755 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:50.751+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:50.755 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:50.751+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:50.756 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:50.751+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:50.756 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:50.751+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:50.886 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:50.883+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:50.886 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:50.883+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:50.887 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:50.883+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:50.887 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:50.883+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:50.944 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:50.939+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:50.944 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:50.939+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:50.945 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:50.939+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:50.945 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:50.939+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:50.945 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:50.939+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:51.446 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:51.443+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:51.447 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:51.443+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:51.447 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:51.443+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:51.447 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:51.443+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:51.758 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:51.747+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:51.758 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:51.747+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:51.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:51.747+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:51.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:51.747+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:51.846 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:51.843+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:51.847 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:51.843+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:51.847 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:51.843+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:51.847 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:51.843+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:51.945 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:51.939+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:51.945 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:51.939+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:51.946 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:51.939+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:51.946 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:51.939+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:51.946 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:51.939+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:52.434 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:52.431+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:52.434 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:52.431+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:52.435 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:52.431+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:52.435 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:52.431+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:52.784 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:52.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:52.785 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:52.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:52.785 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:52.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:52.785 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:52.779+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:52.895 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:52.895+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:52.895 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:52.895+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:52.896 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:52.895+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:52.896 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:52.895+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:52.995 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:52.994+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:52.995 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:52.994+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:52.996 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:52.994+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:52.996 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:52.994+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:52.996 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:52.994+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:53.225 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~40s 2020-05-18T22:10:53.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~40s 2020-05-18T22:10:53.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~40s 2020-05-18T22:10:53.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~40s 2020-05-18T22:10:53.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~40s 2020-05-18T22:10:53.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~40s 2020-05-18T22:10:53.227 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~40s 2020-05-18T22:10:53.227 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~40s 2020-05-18T22:10:53.473 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:53.466+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:53.473 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:53.466+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:53.473 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:53.466+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:53.474 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:53.466+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:53.537 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:10:53.534+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5764 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:10:53.818 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:53.814+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:53.819 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:53.814+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:53.819 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:53.814+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:53.819 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:53.814+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:53.900 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:53.894+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:53.901 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:53.894+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:53.901 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:53.894+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:53.901 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:53.894+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:54.042 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:54.038+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:54.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:54.038+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:54.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:54.038+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:54.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:54.038+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:54.044 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:54.038+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:54.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:54.486+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:54.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:54.486+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:54.494 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:54.486+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:54.494 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:54.486+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:54.803 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:54.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:54.804 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:54.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:54.804 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:54.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:54.804 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:54.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:54.890 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:54.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:54.891 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:54.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:54.891 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:54.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:54.891 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:54.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:54.995 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:54.994+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:54.996 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:54.994+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:54.996 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:54.994+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:54.996 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:54.994+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:54.996 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:54.994+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:55.461 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:55.458+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:55.462 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:55.458+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:55.462 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:55.458+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:55.462 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:55.458+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:55.770 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:55.766+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:55.771 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:55.766+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:55.771 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:55.766+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:55.772 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:55.766+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:55.912 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:55.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:55.912 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:55.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:55.913 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:55.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:55.913 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:55.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:56.038 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:56.034+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:56.038 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:56.034+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:56.039 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:56.034+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:56.039 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:56.034+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:56.040 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:56.034+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:56.425 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:56.418+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:56.426 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:56.418+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:56.426 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:56.418+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:56.426 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:56.418+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:56.806 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:56.802+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:56.807 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:56.802+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:56.807 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:56.802+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:56.807 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:56.802+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:56.944 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:56.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:56.945 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:56.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:56.945 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:56.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:56.945 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:56.942+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:57.046 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:57.042+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:57.046 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:57.042+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:57.046 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:57.042+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:57.047 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:57.042+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:57.047 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:57.042+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:57.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:57.382+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:57.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:57.382+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:57.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:57.382+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:57.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:57.382+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:57.761 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:57.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:57.762 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:57.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:57.762 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:57.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:57.762 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:57.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:57.915 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:57.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:57.916 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:57.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:57.916 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:57.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:57.916 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:57.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:58.061 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:58.058+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:58.061 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:58.058+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:58.061 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:58.058+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:58.062 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:58.058+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:58.062 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:58.058+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:58.391 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:58.386+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:58.391 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:58.386+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:58.391 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:58.386+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:58.392 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:58.386+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:58.539 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:10:58.534+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5799 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:10:58.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:58.762+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:58.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:58.762+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:58.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:58.762+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:58.768 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:58.762+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:58.934 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~46s 2020-05-18T22:10:58.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~46s 2020-05-18T22:10:58.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~46s 2020-05-18T22:10:58.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~46s 2020-05-18T22:10:58.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~46s 2020-05-18T22:10:58.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~46s 2020-05-18T22:10:58.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~46s 2020-05-18T22:10:58.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~46s 2020-05-18T22:10:58.942 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:58.938+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:58.943 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:58.938+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:58.943 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:58.938+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:58.943 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:58.938+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:59.018 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:59.010+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:59.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:59.010+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:59.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:59.010+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:59.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:59.010+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:10:59.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:10:59.010+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:10:59.424 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:59.422+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:59.425 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:59.422+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:59.425 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:59.422+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:59.425 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:10:59.422+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:10:59.773 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:59.766+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:59.773 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:59.766+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:59.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:59.766+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:59.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:10:59.766+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:10:59.980 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:59.974+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:59.980 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:59.974+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:59.980 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:59.974+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:10:59.980 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:10:59.974+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:00.012 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:00.006+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:00.012 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:00.006+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:00.012 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:00.006+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:00.013 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:00.006+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:00.013 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:00.006+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:00.449 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:00.446+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:00.449 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:00.446+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:00.450 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:00.446+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:00.450 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:00.446+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:00.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:00.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:00.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:00.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:00.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:00.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:00.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:00.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:00.952 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:00.950+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:00.952 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:00.950+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:00.953 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:00.950+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:00.953 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:00.950+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:01.015 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:01.010+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:01.015 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:01.010+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:01.016 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:01.010+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:01.016 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:01.010+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:01.016 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:01.010+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:01.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:01.486+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:01.488 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:01.486+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:01.488 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:01.486+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:01.488 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:01.486+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:01.809 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:01.806+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:01.809 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:01.806+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:01.809 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:01.806+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:01.809 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:01.806+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:01.994 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:01.990+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:01.994 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:01.990+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:01.994 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:01.990+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:01.994 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:01.990+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:02.005 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:01.998+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:02.005 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:01.998+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:02.006 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:01.998+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:02.006 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:01.998+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:02.006 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:01.998+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:02.515 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:02.510+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:02.516 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:02.510+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:02.516 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:02.510+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:02.516 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:02.510+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:02.784 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:02.782+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:02.785 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:02.782+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:02.785 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:02.782+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:02.786 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:02.782+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:03.015 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:03.014+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:03.016 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:03.014+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:03.016 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:03.014+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:03.016 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:03.014+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:03.019 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:03.014+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:03.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:03.014+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:03.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:03.014+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:03.020 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:03.014+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:03.021 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:03.014+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:03.540 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:11:03.538+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5836 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:11:03.549 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:03.538+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:03.550 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:03.538+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:03.550 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:03.538+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:03.550 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:03.538+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:03.762 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:03.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:03.763 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:03.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:03.763 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:03.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:03.763 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:03.758+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:03.974 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:03.966+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:03.974 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:03.966+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:03.975 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:03.970+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:03.975 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:03.970+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:04.059 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:04.054+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:04.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:04.054+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:04.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:04.054+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:04.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:04.054+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:04.060 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:04.054+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:04.544 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:04.542+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:04.544 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:04.542+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:04.545 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:04.542+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:04.545 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:04.542+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:04.646 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~51s 2020-05-18T22:11:04.646 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~51s 2020-05-18T22:11:04.647 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~51s 2020-05-18T22:11:04.647 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~51s 2020-05-18T22:11:04.647 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~51s 2020-05-18T22:11:04.647 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~51s 2020-05-18T22:11:04.648 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~51s 2020-05-18T22:11:04.648 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~51s 2020-05-18T22:11:04.735 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:04.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:04.735 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:04.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:04.736 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:04.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:04.736 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:04.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:04.935 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:04.930+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:04.936 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:04.930+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:04.936 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:04.930+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:04.937 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:04.930+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:05.066 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:05.062+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:05.066 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:05.062+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:05.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:05.062+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:05.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:05.062+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:05.067 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:05.062+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:05.523 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:05.518+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:05.523 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:05.518+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:05.524 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:05.518+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:05.524 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:05.518+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:05.784 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:05.778+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:05.784 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:05.778+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:05.784 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:05.778+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:05.785 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:05.778+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:05.935 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:05.930+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:05.935 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:05.930+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:05.936 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:05.930+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:05.936 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:05.930+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:06.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:06.038+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:06.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:06.038+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:06.043 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:06.038+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:06.044 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:06.038+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:06.044 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:06.038+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:06.533 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:06.530+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:06.533 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:06.530+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:06.534 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:06.530+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:06.534 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:06.530+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:06.744 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:06.742+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:06.744 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:06.742+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:06.744 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:06.742+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:06.745 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:06.742+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:06.916 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:06.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:06.916 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:06.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:06.916 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:06.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:06.917 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:06.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:07.061 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:07.058+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:07.062 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:07.058+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:07.062 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:07.058+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:07.062 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:07.058+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:07.063 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:07.058+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:07.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:07.502+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:07.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:07.502+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:07.507 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:07.502+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:07.508 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:07.502+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:07.736 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:07.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:07.737 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:07.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:07.737 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:07.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:07.737 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:07.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:07.921 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:07.918+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:07.921 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:07.918+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:07.921 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:07.918+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:07.921 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:07.918+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:08.079 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:08.078+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:08.080 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:08.078+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:08.080 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:08.078+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:08.080 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:08.078+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:08.081 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:08.078+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:08.509 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:08.502+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:08.509 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:08.502+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:08.510 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:08.502+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:08.510 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:08.502+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:08.542 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:11:08.538+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5870 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:11:08.771 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:08.770+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:08.772 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:08.770+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:08.772 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:08.770+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:08.772 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:08.770+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:08.909 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:08.906+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:08.909 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:08.906+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:08.909 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:08.906+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:08.910 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:08.906+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:09.079 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:09.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:09.080 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:09.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:09.080 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:09.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:09.080 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:09.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:09.080 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:09.074+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:09.529 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:09.522+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:09.529 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:09.522+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:09.529 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:09.522+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:09.530 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:09.522+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:09.738 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:09.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:09.739 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:09.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:09.739 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:09.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:09.739 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:09.734+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:09.887 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:09.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:09.888 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:09.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:09.888 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:09.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:09.888 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:09.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:10.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:10.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:10.079 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:10.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:10.079 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:10.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:10.079 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:10.074+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:10.079 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:10.074+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:10.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~57s 2020-05-18T22:11:10.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~57s 2020-05-18T22:11:10.354 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~57s 2020-05-18T22:11:10.354 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~57s 2020-05-18T22:11:10.354 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~57s 2020-05-18T22:11:10.354 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~57s 2020-05-18T22:11:10.354 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~57s 2020-05-18T22:11:10.355 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~57s 2020-05-18T22:11:10.544 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:10.542+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:10.544 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:10.542+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:10.544 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:10.542+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:10.544 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:10.542+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:10.779 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:10.774+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:10.779 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:10.774+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:10.779 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:10.774+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:10.780 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:10.774+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:10.858 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:10.854+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:10.858 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:10.854+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:10.859 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:10.854+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:10.859 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:10.854+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:11.103 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:11.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:11.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:11.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:11.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:11.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:11.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:11.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:11.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:11.098+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:11.537 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:11.534+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:11.537 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:11.534+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:11.538 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:11.534+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:11.538 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:11.534+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:11.810 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:11.806+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:11.811 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:11.806+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:11.811 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:11.806+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:11.811 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:11.806+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:11.837 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:11.830+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:11.837 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:11.830+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:11.837 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:11.830+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:11.837 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:11.830+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:12.131 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:12.130+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:12.132 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:12.130+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:12.132 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:12.130+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:12.132 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:12.130+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:12.133 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:12.130+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:12.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:12.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:12.494 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:12.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:12.494 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:12.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:12.494 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:12.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:12.560 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:11:12.561 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:11:12.816 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:12.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:12.817 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:12.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:12.817 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:12.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:12.817 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:12.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:12.822 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:12.818+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:12.823 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:12.818+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:12.823 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:12.818+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:12.823 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:12.818+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:13.124 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:13.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:13.124 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:13.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:13.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:13.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:13.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:13.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:13.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:13.122+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:13.540 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:13.534+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:13.540 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:13.534+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:13.540 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:13.534+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:13.540 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:13.534+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:13.543 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:11:13.538+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5906 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:11:13.778 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:13.778+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:13.779 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:13.778+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:13.779 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:13.778+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:13.779 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:13.778+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:13.869 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:13.866+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:13.869 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:13.866+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:13.869 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:13.866+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:13.870 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:13.866+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:14.104 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:14.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:14.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:14.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:14.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:14.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:14.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:14.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:14.105 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:14.098+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:14.499 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:14.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:14.500 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:14.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:14.500 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:14.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:14.500 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:14.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:14.793 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:14.790+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:14.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:14.790+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:14.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:14.790+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:14.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:14.790+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:14.874 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:14.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:14.875 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:14.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:14.875 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:14.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:14.875 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:14.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:15.086 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:15.078+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:15.086 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:15.078+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:15.086 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:15.078+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:15.086 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:15.078+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:15.087 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:15.078+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:15.500 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:15.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:15.501 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:15.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:15.501 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:15.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:15.501 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:15.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:15.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:11:15.614 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:11:15.837 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:15.834+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:15.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:15.834+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:15.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:15.834+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:15.839 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:15.834+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:15.852 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:15.846+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:15.852 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:15.846+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:15.852 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:15.846+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:15.852 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:15.846+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:16.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~63s 2020-05-18T22:11:16.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~63s 2020-05-18T22:11:16.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~63s 2020-05-18T22:11:16.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~63s 2020-05-18T22:11:16.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~63s 2020-05-18T22:11:16.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~63s 2020-05-18T22:11:16.062 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~63s 2020-05-18T22:11:16.062 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~63s 2020-05-18T22:11:16.089 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:16.086+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:16.089 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:16.086+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:16.089 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:16.086+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:16.089 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:16.086+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:16.089 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:16.086+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:16.464 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:16.462+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:16.465 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:16.462+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:16.465 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:16.462+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:16.465 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:16.462+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:16.849 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:16.846+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:16.849 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:16.846+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:16.850 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:16.846+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:16.850 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:16.846+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:16.859 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:16.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:16.860 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:16.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:16.860 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:16.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:16.860 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:16.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:17.125 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:17.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:17.126 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:17.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:17.126 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:17.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:17.126 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:17.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:17.127 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:17.122+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:17.486 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:17.482+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:17.486 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:17.482+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:17.486 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:17.482+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:17.486 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:17.482+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:17.874 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:17.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:17.875 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:17.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:17.875 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:17.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:17.875 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:17.874+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:17.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:17.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:17.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:17.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:17.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:17.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:17.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:17.882+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:18.142 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:18.138+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:18.142 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:18.138+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:18.143 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:18.138+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:18.143 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:18.138+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:18.143 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:18.138+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:18.534 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:18.534+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:18.535 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:18.534+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:18.535 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:18.534+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:18.535 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:18.534+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:18.545 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:11:18.542+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5941 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:11:18.829 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:18.826+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:18.829 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:18.826+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:18.830 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:18.826+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:18.830 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:18.826+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:18.868 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:18.866+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:18.869 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:18.866+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:18.869 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:18.866+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:18.869 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:18.866+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:19.137 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:19.134+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:19.138 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:19.134+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:19.138 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:19.134+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:19.138 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:19.134+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:19.138 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:19.134+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:19.566 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:19.566+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:19.567 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:19.566+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:19.567 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:19.566+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:19.567 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:19.566+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:19.837 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:19.834+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:19.838 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:19.834+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:19.838 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:19.834+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:19.838 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:19.834+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:19.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:19.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:19.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:19.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:19.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:19.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:19.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:19.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:20.151 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:20.150+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:20.152 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:20.150+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:20.152 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:20.150+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:20.152 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:20.150+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:20.152 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:20.150+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:20.552 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:20.550+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:20.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:20.550+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:20.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:20.550+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:20.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:20.550+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:20.816 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:20.814+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:20.816 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:20.814+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:20.816 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:20.814+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:20.817 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:20.814+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:20.826 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:20.822+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:20.826 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:20.822+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:20.827 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:20.822+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:20.827 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:20.826+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:21.164 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:21.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:21.164 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:21.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:21.164 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:21.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:21.165 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:21.162+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:21.165 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:21.162+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:21.559 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:21.558+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:21.560 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:21.558+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:21.560 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:21.558+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:21.560 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:21.558+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:21.771 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~68s 2020-05-18T22:11:21.771 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~68s 2020-05-18T22:11:21.771 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~68s 2020-05-18T22:11:21.772 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~68s 2020-05-18T22:11:21.772 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~68s 2020-05-18T22:11:21.772 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~68s 2020-05-18T22:11:21.772 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~68s 2020-05-18T22:11:21.772 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~68s 2020-05-18T22:11:21.833 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:21.830+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:21.833 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:21.830+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:21.833 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:21.830+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:21.834 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:21.830+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:21.840 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:21.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:21.840 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:21.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:21.840 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:21.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:21.841 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:21.838+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:22.182 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:22.182+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:22.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:22.182+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:22.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:22.182+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:22.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:22.182+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:22.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:22.182+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:22.567 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:22.566+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:22.568 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:22.566+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:22.568 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:22.566+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:22.568 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:22.566+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:22.810 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:22.806+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:22.811 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:22.806+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:22.811 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:22.806+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:22.811 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:22.806+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:22.826 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:22.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:22.826 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:22.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:22.826 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:22.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:22.827 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:22.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:23.227 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:23.226+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:23.228 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:23.226+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:23.228 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:23.226+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:23.228 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:23.226+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:23.228 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:23.226+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:23.547 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:11:23.546+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 5977 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:11:23.548 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:23.546+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:23.548 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:23.546+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:23.549 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:23.546+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:23.549 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:23.546+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:23.815 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:23.814+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:23.816 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:23.814+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:23.816 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:23.814+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:23.817 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:23.814+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:23.847 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:23.846+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:23.847 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:23.846+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:23.848 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:23.846+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:23.848 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:23.846+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:24.212 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:24.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:24.213 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:24.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:24.213 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:24.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:24.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:24.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:24.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:24.210+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:24.499 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:24.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:24.500 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:24.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:24.500 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:24.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:24.500 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:24.498+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:24.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:24.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:24.849 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:24.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:24.849 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:24.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:24.849 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:24.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:24.869 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:24.866+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:24.869 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:24.866+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:24.869 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:24.866+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:24.870 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:24.866+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:25.236 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:25.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:25.237 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:25.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:25.237 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:25.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:25.237 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:25.234+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:25.237 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:25.234+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:25.512 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:25.510+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:25.512 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:25.510+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:25.512 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:25.510+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:25.513 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:25.510+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:25.856 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:25.854+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:25.856 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:25.854+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:25.856 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:25.854+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:25.856 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:25.854+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:25.913 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:25.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:25.913 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:25.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:25.913 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:25.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:25.914 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:25.910+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:26.213 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:26.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:26.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:26.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:26.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:26.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:26.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:26.210+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:26.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:26.210+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:26.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:26.502+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:26.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:26.502+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:26.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:26.502+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:26.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:26.502+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:26.826 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:26.826+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:26.827 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:26.826+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:26.827 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:26.826+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:26.828 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:26.826+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:26.918 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:26.914+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:26.919 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:26.914+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:26.919 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:26.914+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:26.919 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:26.914+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:27.173 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:27.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:27.173 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:27.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:27.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:27.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:27.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:27.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:27.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:27.170+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:27.481 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~74s 2020-05-18T22:11:27.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~74s 2020-05-18T22:11:27.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~74s 2020-05-18T22:11:27.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~74s 2020-05-18T22:11:27.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~74s 2020-05-18T22:11:27.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~74s 2020-05-18T22:11:27.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~74s 2020-05-18T22:11:27.484 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~74s 2020-05-18T22:11:27.491 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:27.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:27.491 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:27.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:27.492 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:27.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:27.492 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:27.490+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:27.798 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:27.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:27.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:27.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:27.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:27.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:27.800 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:27.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:27.950 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:27.950+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:27.951 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:27.950+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:27.951 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:27.950+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:27.951 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:27.950+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:28.139 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:28.138+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:28.140 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:28.138+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:28.140 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:28.138+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:28.140 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:28.138+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:28.141 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:28.138+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:28.520 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:28.518+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:28.521 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:28.518+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:28.521 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:28.518+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:28.521 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:28.518+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:28.549 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:11:28.546+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6006 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:11:28.848 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:28.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:28.849 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:28.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:28.849 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:28.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:28.849 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:28.846+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:28.906 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:28.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:28.906 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:28.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:28.906 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:28.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:28.907 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:28.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:29.126 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:29.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:29.126 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:29.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:29.127 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:29.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:29.127 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:29.122+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:29.127 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:29.122+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:29.472 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:29.470+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:29.472 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:29.470+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:29.473 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:29.470+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:29.473 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:29.470+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:29.852 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:29.850+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:29.852 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:29.850+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:29.852 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:29.850+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:29.853 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:29.850+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:29.906 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:29.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:29.906 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:29.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:29.907 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:29.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:29.907 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:29.902+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:30.100 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:30.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:30.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:30.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:30.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:30.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:30.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:30.098+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:30.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:30.098+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:30.452 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:30.450+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:30.452 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:30.450+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:30.453 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:30.450+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:30.453 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:30.450+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:30.805 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:30.802+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:30.806 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:30.802+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:30.806 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:30.802+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:30.806 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:30.802+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:30.928 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:30.926+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:30.929 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:30.926+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:30.929 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:30.926+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:30.929 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:30.926+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:31.111 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:31.110+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:31.111 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:31.110+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:31.111 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:31.110+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:31.112 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:31.110+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:31.112 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:31.110+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:31.406 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:31.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:31.406 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:31.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:31.407 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:31.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:31.407 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:31.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:31.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:31.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:31.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:31.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:31.799 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:31.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:31.800 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:31.798+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:31.900 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:31.898+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:31.900 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:31.898+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:31.900 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:31.898+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:31.901 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:31.898+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:32.119 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:32.118+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:32.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:32.118+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:32.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:32.118+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:32.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:32.118+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:32.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:32.118+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:32.406 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:32.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:32.406 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:32.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:32.407 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:32.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:32.407 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:32.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:32.784 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:32.782+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:32.785 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:32.782+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:32.785 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:32.782+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:32.785 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:32.782+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:32.863 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:32.862+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:32.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:32.862+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:32.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:32.862+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:32.864 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:32.862+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:33.093 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:33.090+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:33.093 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:33.090+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:33.093 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:33.090+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:33.094 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:33.090+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:33.094 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:33.090+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:33.196 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~80s 2020-05-18T22:11:33.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~80s 2020-05-18T22:11:33.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~80s 2020-05-18T22:11:33.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~80s 2020-05-18T22:11:33.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~80s 2020-05-18T22:11:33.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~80s 2020-05-18T22:11:33.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~80s 2020-05-18T22:11:33.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~80s 2020-05-18T22:11:33.405 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:33.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:33.405 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:33.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:33.406 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:33.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:33.406 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:33.402+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:33.550 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:11:33.550+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6042 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:11:33.788 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:33.786+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:33.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:33.786+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:33.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:33.786+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:33.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:33.786+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:33.886 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:33.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:33.887 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:33.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:33.887 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:33.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:33.887 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:33.886+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:34.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:34.134+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:34.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:34.134+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:34.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:34.134+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:34.136 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:34.134+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:34.136 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:34.134+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:34.448 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:34.446+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:34.448 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:34.446+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:34.449 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:34.446+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:34.449 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:34.446+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:34.837 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:34.834+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:34.837 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:34.834+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:34.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:34.834+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:34.838 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:34.834+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:34.921 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:34.918+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:34.921 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:34.918+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:34.922 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:34.918+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:34.922 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:34.918+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:35.169 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:35.166+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:35.169 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:35.166+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:35.169 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:35.166+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:35.170 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:35.166+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:35.170 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:35.166+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:35.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:35.438+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:35.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:35.438+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:35.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:35.438+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:35.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:35.438+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:35.812 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:35.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:35.812 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:35.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:35.813 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:35.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:35.813 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:35.810+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:35.938 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:35.938+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:35.939 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:35.938+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:35.939 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:35.938+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:35.939 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:35.938+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:36.200 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:36.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:36.201 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:36.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:36.201 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:36.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:36.201 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:36.198+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:36.201 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:36.198+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:36.437 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:36.434+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:36.437 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:36.434+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:36.437 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:36.434+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:36.438 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:36.434+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:36.856 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:36.854+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:36.857 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:36.854+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:36.857 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:36.854+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:36.857 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:36.854+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:36.890 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:36.890+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:36.890 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:36.890+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:36.890 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:36.890+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:36.891 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:36.890+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:37.173 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:37.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:37.173 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:37.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:37.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:37.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:37.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:37.170+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:37.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:37.170+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:37.462 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:37.458+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:37.462 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:37.458+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:37.462 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:37.458+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:37.463 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:37.462+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:37.844 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:37.842+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:37.845 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:37.842+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:37.845 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:37.842+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:37.845 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:37.842+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:37.861 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:37.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:37.862 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:37.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:37.862 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:37.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:37.862 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:37.858+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:38.134 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:38.133+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:38.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:38.133+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:38.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:38.133+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:38.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:38.133+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:38.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:38.133+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:38.474 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:38.473+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:38.475 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:38.473+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:38.475 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:38.473+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:38.475 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:38.473+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:38.552 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:11:38.549+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6077 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:11:38.838 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:38.837+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:38.839 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:38.837+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:38.839 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:38.837+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:38.839 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:38.837+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:38.904 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~86s 2020-05-18T22:11:38.904 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~86s 2020-05-18T22:11:38.904 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~86s 2020-05-18T22:11:38.904 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~86s 2020-05-18T22:11:38.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~86s 2020-05-18T22:11:38.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~86s 2020-05-18T22:11:38.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~86s 2020-05-18T22:11:38.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~86s 2020-05-18T22:11:38.906 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:38.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:38.906 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:38.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:38.907 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:38.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:38.907 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:38.905+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:39.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:39.117+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:39.120 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:39.117+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:39.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:39.117+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:39.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:39.117+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:39.121 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:39.117+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:39.446 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:39.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:39.446 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:39.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:39.447 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:39.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:39.447 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:39.445+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:39.844 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:39.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:39.844 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:39.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:39.845 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:39.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:39.845 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:39.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:39.904 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:39.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:39.905 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:39.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:39.905 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:39.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:39.905 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:39.901+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:40.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:40.097+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:40.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:40.097+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:40.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:40.097+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:40.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:40.097+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:40.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:40.097+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:40.467 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:40.465+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:40.467 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:40.465+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:40.467 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:40.465+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:40.468 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:40.465+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:40.808 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:40.805+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:40.808 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:40.805+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:40.808 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:40.805+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:40.808 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:40.805+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:40.932 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:40.929+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:40.932 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:40.929+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:40.932 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:40.929+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:40.933 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:40.929+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:41.126 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:41.125+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:41.127 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:41.125+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:41.127 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:41.125+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:41.127 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:41.125+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:41.127 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:41.125+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:41.491 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:41.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:41.492 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:41.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:41.492 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:41.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:41.492 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:41.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:41.848 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:41.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:41.849 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:41.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:41.849 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:41.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:41.849 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:41.845+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:41.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:41.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:41.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:41.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:41.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:41.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:41.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:41.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:42.078 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:42.077+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:42.079 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:42.077+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:42.079 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:42.077+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:42.079 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:42.077+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:42.079 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:42.077+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:42.480 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:42.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:42.481 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:42.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:42.481 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:42.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:42.481 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:42.477+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:42.857 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:42.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:42.857 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:42.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:42.857 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:42.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:42.858 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:42.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:42.960 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:42.957+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:42.960 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:42.957+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:42.961 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:42.957+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:42.961 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:42.957+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:43.116 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:43.113+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:43.117 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:43.113+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:43.117 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:43.113+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:43.117 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:43.113+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:43.117 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:43.113+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:43.465 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:43.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:43.466 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:43.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:43.466 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:43.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:43.466 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:43.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:43.554 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:11:43.549+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6114 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:11:43.862 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:43.861+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:43.863 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:43.861+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:43.863 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:43.861+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:43.863 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:43.861+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:43.915 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:43.913+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:43.915 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:43.913+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:43.916 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:43.913+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:43.916 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:43.913+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:44.073 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:44.069+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:44.074 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:44.069+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:44.074 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:44.069+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:44.074 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:44.069+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:44.074 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:44.069+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:44.439 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:44.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:44.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:44.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:44.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:44.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:44.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:44.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:44.610 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~91s 2020-05-18T22:11:44.610 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~91s 2020-05-18T22:11:44.611 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~91s 2020-05-18T22:11:44.611 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~91s 2020-05-18T22:11:44.611 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~91s 2020-05-18T22:11:44.611 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~91s 2020-05-18T22:11:44.611 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~91s 2020-05-18T22:11:44.612 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~91s 2020-05-18T22:11:44.818 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:44.813+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:44.819 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:44.813+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:44.819 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:44.813+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:44.819 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:44.813+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:44.964 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:44.961+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:44.965 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:44.961+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:44.965 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:44.961+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:44.965 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:44.961+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:45.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:45.097+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:45.101 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:45.097+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:45.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:45.097+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:45.102 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:45.097+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:45.103 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:45.097+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:45.436 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:45.433+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:45.436 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:45.433+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:45.436 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:45.433+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:45.436 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:45.433+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:45.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:11:45.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:11:45.856 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:45.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:45.857 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:45.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:45.857 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:45.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:45.857 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:45.853+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:46.007 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:46.005+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:46.007 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:46.005+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:46.007 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:46.005+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:46.007 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:46.005+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:46.134 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:46.129+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:46.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:46.129+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:46.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:46.129+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:46.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:46.129+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:46.135 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:46.129+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:46.464 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:46.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:46.464 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:46.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:46.465 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:46.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:46.465 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:46.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:46.893 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:46.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:46.894 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:46.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:46.894 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:46.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:46.894 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:46.889+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:47.006 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:47.005+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:47.007 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:47.005+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:47.007 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:47.005+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:47.007 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:47.005+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:47.166 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:47.161+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:47.166 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:47.161+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:47.166 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:47.161+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:47.166 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:47.161+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:47.167 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:47.161+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:47.460 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:47.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:47.461 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:47.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:47.461 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:47.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:47.461 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:47.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:47.914 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:47.913+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:47.915 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:47.913+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:47.915 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:47.913+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:47.915 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:47.913+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:47.972 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:47.969+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:47.973 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:47.969+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:47.973 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:47.969+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:47.973 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:47.969+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:48.192 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:48.189+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:48.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:48.189+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:48.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:48.189+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:48.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:48.189+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:48.193 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:48.189+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:48.477 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:48.473+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:48.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:48.473+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:48.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:48.473+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:48.478 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:48.473+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:48.556 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:11:48.553+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6148 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:11:48.670 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:11:48.670 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:11:48.937 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:48.933+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:48.937 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:48.933+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:48.938 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:48.933+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:48.938 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:48.933+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:48.944 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:48.941+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:48.944 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:48.941+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:48.944 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:48.941+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:48.944 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:48.941+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:49.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:49.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:49.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:49.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:49.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:49.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:49.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:49.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:49.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:49.217+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:49.501 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:49.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:49.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:49.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:49.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:49.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:49.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:49.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:49.924 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:49.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:49.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:49.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:49.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:49.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:49.925 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:49.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:49.968 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:49.965+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:49.968 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:49.965+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:49.969 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:49.965+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:49.969 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:49.965+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:50.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:50.209+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:50.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:50.209+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:50.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:50.209+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:50.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:50.209+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:50.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:50.209+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:50.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~97s 2020-05-18T22:11:50.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~97s 2020-05-18T22:11:50.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~97s 2020-05-18T22:11:50.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~97s 2020-05-18T22:11:50.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~97s 2020-05-18T22:11:50.320 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~97s 2020-05-18T22:11:50.320 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~97s 2020-05-18T22:11:50.320 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~97s 2020-05-18T22:11:50.455 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:50.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:50.455 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:50.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:50.456 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:50.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:50.456 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:50.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:50.879 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:50.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:50.880 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:50.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:50.880 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:50.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:50.880 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:50.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:50.994 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:50.989+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:50.994 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:50.989+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:50.995 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:50.989+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:50.995 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:50.989+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:51.180 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:51.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:51.180 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:51.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:51.181 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:51.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:51.181 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:51.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:51.181 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:51.177+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:51.438 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:51.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:51.439 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:51.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:51.439 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:51.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:51.439 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:51.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:51.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:51.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:51.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:51.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:51.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:51.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:51.881 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:51.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:51.992 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:51.989+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:51.993 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:51.989+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:51.993 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:51.989+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:51.993 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:51.989+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:52.171 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:52.169+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:52.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:52.169+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:52.172 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:52.169+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:52.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:52.169+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:52.174 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:52.169+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:52.476 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:52.473+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:52.477 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:52.473+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:52.477 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:52.473+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:52.477 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:52.473+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:52.855 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:52.853+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:52.856 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:52.853+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:52.856 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:52.853+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:52.856 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:52.853+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:53.019 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:53.017+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:53.182 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:53.017+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:53.183 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:53.017+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:53.183 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:53.017+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:53.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:53.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:53.221 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:53.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:53.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:53.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:53.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:53.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:53.222 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:53.217+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:53.432 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:53.429+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:53.433 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:53.429+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:53.433 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:53.429+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:53.433 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:53.429+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:53.557 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:11:53.553+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6185 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:11:53.816 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:53.813+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:53.817 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:53.813+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:53.817 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:53.813+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:53.817 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:53.813+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:54.024 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:54.021+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:54.025 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:54.021+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:54.025 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:54.021+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:54.025 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:54.021+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:54.230 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:54.229+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:54.231 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:54.229+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:54.231 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:54.229+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:54.231 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:54.229+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:54.231 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:54.229+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:54.439 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:54.429+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:54.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:54.429+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:54.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:54.429+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:54.440 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:54.429+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:54.844 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:54.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:54.845 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:54.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:54.845 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:54.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:54.845 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:54.841+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:55.027 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:55.025+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:55.028 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:55.025+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:55.028 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:55.025+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:55.028 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:55.025+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:55.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:55.213+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:55.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:55.213+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:55.215 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:55.213+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:55.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:55.213+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:55.216 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:55.213+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:55.454 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:55.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:55.455 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:55.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:55.455 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:55.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:55.455 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:55.453+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:55.863 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:55.861+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:55.863 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:55.861+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:55.863 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:55.861+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:55.864 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:55.861+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:56.016 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:56.013+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:56.017 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:56.013+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:56.017 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:56.013+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:56.017 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:56.013+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:56.026 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~103s 2020-05-18T22:11:56.026 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~103s 2020-05-18T22:11:56.026 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~103s 2020-05-18T22:11:56.027 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~103s 2020-05-18T22:11:56.027 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~103s 2020-05-18T22:11:56.027 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~103s 2020-05-18T22:11:56.027 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~103s 2020-05-18T22:11:56.027 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~103s 2020-05-18T22:11:56.223 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:56.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:56.224 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:56.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:56.224 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:56.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:56.225 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:56.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:56.225 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:56.217+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:56.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:56.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:56.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:56.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:56.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:56.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:56.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:56.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:56.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:56.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:56.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:56.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:56.913 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:56.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:56.914 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:56.909+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:56.986 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:56.985+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:56.987 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:56.985+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:56.987 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:56.985+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:56.987 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:56.985+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:57.207 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:57.205+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:57.207 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:57.205+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:57.208 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:57.205+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:57.208 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:57.205+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:57.208 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:57.205+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:57.430 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:57.429+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:57.430 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:57.429+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:57.430 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:57.429+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:57.430 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:57.429+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:57.919 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:57.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:57.919 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:57.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:57.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:57.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:57.920 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:57.917+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:57.991 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:57.989+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:57.991 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:57.989+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:57.991 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:57.989+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:57.992 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:57.989+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:58.191 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:58.189+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:58.191 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:58.189+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:58.192 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:58.189+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:58.192 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:58.189+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:58.192 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:58.189+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:58.469 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:58.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:58.469 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:58.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:58.469 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:58.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:58.470 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:58.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:58.559 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:11:58.557+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6220 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:11:58.877 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:58.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:58.877 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:58.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:58.878 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:58.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:58.878 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:58.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:58.974 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:58.969+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:58.974 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:58.969+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:58.974 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:58.969+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:58.975 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:58.969+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:59.218 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:59.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:59.219 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:59.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:59.219 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:59.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:59.219 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:59.217+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:11:59.219 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:11:59.217+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:11:59.423 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:59.421+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:59.423 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:59.421+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:59.423 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:59.421+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:59.424 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:11:59.421+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:11:59.922 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:59.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:59.922 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:59.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:59.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:59.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:59.923 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:11:59.921+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:11:59.940 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:59.937+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:59.941 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:59.937+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:59.941 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:59.937+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:11:59.941 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:11:59.937+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:00.264 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:00.261+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:00.265 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:00.261+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:00.265 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:00.261+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:00.265 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:00.261+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:00.265 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:00.261+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:00.460 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:00.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:00.460 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:00.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:00.460 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:00.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:00.461 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:00.457+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:00.885 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:00.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:00.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:00.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:00.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:00.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:00.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:00.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:00.971 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:00.969+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:00.972 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:00.969+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:00.972 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:00.969+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:00.972 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:00.969+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:01.265 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:01.265+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:01.266 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:01.265+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:01.266 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:01.265+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:01.266 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:01.265+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:01.266 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:01.265+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:01.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:01.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:01.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:01.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:01.499 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:01.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:01.499 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:01.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:01.734 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~108s 2020-05-18T22:12:01.734 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~108s 2020-05-18T22:12:01.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~108s 2020-05-18T22:12:01.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~108s 2020-05-18T22:12:01.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~108s 2020-05-18T22:12:01.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~108s 2020-05-18T22:12:01.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~108s 2020-05-18T22:12:01.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~108s 2020-05-18T22:12:01.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:01.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:01.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:01.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:01.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:01.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:01.886 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:01.881+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:01.937 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:01.933+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:01.937 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:01.933+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:01.938 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:01.933+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:01.938 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:01.933+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:02.224 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:02.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:02.225 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:02.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:02.225 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:02.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:02.226 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:02.221+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:02.226 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:02.221+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:02.465 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:02.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:02.466 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:02.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:02.466 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:02.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:02.466 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:02.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:02.888 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:02.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:02.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:02.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:02.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:02.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:02.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:02.885+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:02.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:02.897+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:02.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:02.897+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:02.902 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:02.897+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:02.903 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:02.897+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:03.217 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:03.213+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:03.217 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:03.213+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:03.217 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:03.213+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:03.218 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:03.213+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:03.218 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:03.213+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:03.512 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:03.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:03.512 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:03.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:03.513 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:03.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:03.513 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:03.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:03.560 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:12:03.557+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6257 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:12:03.880 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:03.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:03.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:03.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:03.889 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:03.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:03.890 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:03.877+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:03.955 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:03.949+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:03.956 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:03.949+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:03.956 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:03.949+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:03.956 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:03.949+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:04.213 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:04.209+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:04.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:04.209+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:04.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:04.209+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:04.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:04.209+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:04.214 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:04.209+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:04.548 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:04.545+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:04.549 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:04.545+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:04.549 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:04.545+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:04.549 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:04.545+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:04.894 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:04.893+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:04.895 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:04.893+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:04.895 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:04.893+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:04.895 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:04.893+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:04.954 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:04.953+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:04.954 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:04.953+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:04.954 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:04.953+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:04.954 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:04.953+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:05.182 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:05.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:05.182 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:05.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:05.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:05.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:05.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:05.177+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:05.183 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:05.177+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:05.510 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:05.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:05.510 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:05.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:05.511 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:05.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:05.511 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:05.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:05.852 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:05.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:05.852 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:05.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:05.853 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:05.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:05.853 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:05.849+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:05.918 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:05.917+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:05.919 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:05.917+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:05.919 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:05.917+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:05.919 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:05.917+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:06.197 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:06.193+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:06.197 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:06.193+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:06.198 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:06.193+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:06.198 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:06.193+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:06.198 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:06.193+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:06.544 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:06.541+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:06.544 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:06.541+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:06.544 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:06.541+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:06.545 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:06.541+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:06.827 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:06.825+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:06.827 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:06.825+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:06.827 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:06.825+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:06.827 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:06.825+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:06.933 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:06.929+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:06.933 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:06.929+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:06.934 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:06.929+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:06.934 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:06.929+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:07.201 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:07.197+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:07.201 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:07.197+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:07.201 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:07.197+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:07.201 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:07.197+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:07.202 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:07.197+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:07.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~114s 2020-05-18T22:12:07.441 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~114s 2020-05-18T22:12:07.441 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~114s 2020-05-18T22:12:07.441 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~114s 2020-05-18T22:12:07.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~114s 2020-05-18T22:12:07.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~114s 2020-05-18T22:12:07.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~114s 2020-05-18T22:12:07.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~114s 2020-05-18T22:12:07.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:07.549+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:07.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:07.549+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:07.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:07.549+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:07.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:07.549+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:07.808 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:07.805+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:07.808 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:07.805+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:07.808 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:07.805+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:07.808 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:07.805+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:07.952 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:07.949+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:07.952 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:07.949+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:07.953 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:07.949+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:07.953 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:07.949+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:08.245 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:08.241+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:08.245 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:08.241+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:08.245 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:08.241+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:08.246 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:08.241+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:08.246 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:08.241+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:08.541 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:08.537+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:08.542 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:08.537+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:08.542 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:08.537+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:08.542 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:08.537+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:08.562 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:12:08.557+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6291 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:12:08.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:08.765+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:08.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:08.765+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:08.768 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:08.765+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:08.768 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:08.765+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:08.909 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:08.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:08.910 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:08.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:08.910 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:08.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:08.910 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:08.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:09.289 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:09.285+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:09.289 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:09.285+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:09.289 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:09.285+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:09.290 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:09.285+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:09.290 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:09.285+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:09.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:09.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:09.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:09.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:09.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:09.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:09.493 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:09.489+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:09.800 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:09.797+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:09.800 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:09.797+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:09.801 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:09.797+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:09.801 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:09.797+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:09.868 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:09.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:09.869 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:09.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:09.870 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:09.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:09.870 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:09.865+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:10.298 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:10.297+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:10.299 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:10.297+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:10.299 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:10.297+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:10.299 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:10.297+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:10.299 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:10.297+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:10.514 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:10.513+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:10.514 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:10.513+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:10.515 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:10.513+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:10.515 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:10.513+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:10.819 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:10.817+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:10.819 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:10.817+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:10.819 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:10.817+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:10.820 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:10.817+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:10.905 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:10.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:10.905 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:10.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:10.906 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:10.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:10.906 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:10.905+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:11.316 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:11.313+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:11.317 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:11.313+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:11.317 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:11.313+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:11.317 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:11.313+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:11.317 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:11.313+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:11.522 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:11.521+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:11.522 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:11.521+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:11.522 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:11.521+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:11.522 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:11.521+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:11.793 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:11.789+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:11.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:11.789+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:11.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:11.789+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:11.794 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:11.789+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:11.881 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:11.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:11.881 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:11.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:11.882 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:11.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:11.882 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:11.877+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:12.307 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:12.305+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:12.307 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:12.305+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:12.308 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:12.305+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:12.308 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:12.305+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:12.308 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:12.305+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:12.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:12.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:12.499 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:12.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:12.499 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:12.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:12.499 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:12.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:12.805 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:12.801+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:12.805 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:12.801+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:12.805 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:12.801+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:12.806 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:12.801+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:12.843 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:12.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:12.843 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:12.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:12.843 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:12.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:12.844 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:12.841+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:13.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~120s 2020-05-18T22:12:13.148 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~120s 2020-05-18T22:12:13.148 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~120s 2020-05-18T22:12:13.148 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~120s 2020-05-18T22:12:13.148 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~120s 2020-05-18T22:12:13.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~120s 2020-05-18T22:12:13.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~120s 2020-05-18T22:12:13.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~120s 2020-05-18T22:12:13.355 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:13.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:13.356 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:13.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:13.356 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:13.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:13.356 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:13.353+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:13.356 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:13.353+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:13.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:13.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:13.487 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:13.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:13.488 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:13.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:13.488 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:13.485+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:13.564 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:12:13.561+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6328 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:12:13.796 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:13.793+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:13.797 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:13.793+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:13.797 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:13.793+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:13.797 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:13.793+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:13.825 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:13.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:13.825 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:13.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:13.825 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:13.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:13.825 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:13.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:14.403 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:14.401+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:14.404 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:14.401+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:14.404 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:14.401+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:14.404 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:14.401+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:14.404 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:14.401+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:14.465 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:14.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:14.465 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:14.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:14.466 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:14.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:14.466 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:14.461+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:14.786 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:14.785+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:14.787 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:14.785+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:14.787 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:14.785+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:14.787 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:14.785+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:14.820 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:14.817+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:14.821 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:14.817+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:14.821 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:14.817+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:14.821 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:14.817+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:15.412 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:15.409+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:15.413 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:15.409+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:15.413 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:15.409+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:15.413 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:15.409+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:15.413 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:15.409+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:15.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:15.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:15.441 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:15.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:15.442 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:15.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:15.442 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:15.437+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:15.773 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:15.769+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:15.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:15.769+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:15.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:15.769+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:15.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:15.769+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:15.799 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:15.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:15.800 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:15.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:15.800 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:15.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:15.800 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:15.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:16.416 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:16.413+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:16.417 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:16.413+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:16.417 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:16.413+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:16.417 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:16.413+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:16.424 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:16.421+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:16.424 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:16.421+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:16.424 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:16.421+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:16.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:16.421+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:16.425 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:16.421+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:16.766 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:16.765+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:16.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:16.765+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:16.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:16.765+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:16.767 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:16.765+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:16.823 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:16.821+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:16.823 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:16.821+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:16.823 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:16.821+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:16.824 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:16.821+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:17.388 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:17.385+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:17.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:17.385+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:17.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:17.385+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:17.389 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:17.385+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:17.450 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:17.449+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:17.451 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:17.449+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:17.451 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:17.449+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:17.451 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:17.449+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:17.451 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:17.449+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:17.784 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:17.781+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:17.785 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:17.781+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:17.785 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:17.781+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:17.785 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:17.781+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:17.786 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:17.781+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:17.786 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:17.781+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:17.786 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:17.781+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:17.787 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:17.781+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:18.396 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:18.393+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:18.396 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:18.393+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:18.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:18.393+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:18.397 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:18.393+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:18.485 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:18.485+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:18.486 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:18.485+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:18.486 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:18.485+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:18.486 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:18.485+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:18.486 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:18.485+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:18.566 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:12:18.561+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6363 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:12:18.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:12:18.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:12:18.775 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:18.773+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:18.776 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:18.773+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:18.776 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:18.773+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:18.777 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:18.773+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:18.836 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:18.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:18.836 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:18.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:18.837 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:18.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:18.837 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:18.833+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:18.857 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~126s 2020-05-18T22:12:18.857 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~126s 2020-05-18T22:12:18.857 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~126s 2020-05-18T22:12:18.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~126s 2020-05-18T22:12:18.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~126s 2020-05-18T22:12:18.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~126s 2020-05-18T22:12:18.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~126s 2020-05-18T22:12:18.859 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~126s 2020-05-18T22:12:19.433 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:19.433+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:19.434 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:19.433+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:19.434 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:19.433+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:19.434 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:19.433+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:19.495 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:19.493+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:19.495 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:19.493+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:19.495 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:19.493+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:19.495 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:19.493+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:19.496 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:19.493+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:19.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:19.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:19.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:19.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:19.798 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:19.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:19.799 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:19.797+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:19.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:19.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:19.876 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:19.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:19.877 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:19.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:19.877 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:19.873+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:20.458 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:20.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:20.459 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:20.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:20.459 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:20.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:20.459 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:20.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:20.459 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:20.457+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:20.469 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:20.469+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:20.470 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:20.469+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:20.470 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:20.469+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:20.470 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:20.469+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:20.752 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:20.749+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:20.752 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:20.749+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:20.752 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:20.749+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:20.753 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:20.749+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:20.846 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:20.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:20.846 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:20.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:20.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:20.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:20.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:20.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:21.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:21.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:21.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:21.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:21.460 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:21.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:21.461 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:21.457+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:21.461 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:21.457+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:21.511 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:21.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:21.511 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:21.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:21.511 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:21.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:21.512 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:21.509+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:21.716 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:21.713+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:21.716 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:21.713+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:21.717 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:21.713+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:21.717 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:21.713+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:21.726 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:12:21.726 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:12:21.807 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:21.805+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:21.807 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:21.805+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:21.807 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:21.805+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:21.808 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:21.805+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:22.456 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:22.453+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:22.456 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:22.453+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:22.456 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:22.453+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:22.456 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:22.453+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:22.457 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:22.453+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:22.546 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:22.541+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:22.546 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:22.541+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:22.546 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:22.541+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:22.546 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:22.541+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:22.690 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:22.689+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:22.691 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:22.689+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:22.691 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:22.689+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:22.691 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:22.689+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:22.825 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:22.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:22.825 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:22.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:22.825 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:22.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:22.825 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:22.821+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:23.474 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:23.473+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:23.474 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:23.473+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:23.474 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:23.473+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:23.474 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:23.473+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:23.475 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:23.473+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:23.501 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:23.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:23.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:23.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:23.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:23.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:23.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:23.497+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:23.567 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:12:23.565+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6400 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:12:23.663 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:23.661+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:23.663 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:23.661+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:23.664 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:23.661+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:23.664 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:23.661+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:23.809 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:23.809+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:23.810 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:23.809+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:23.810 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:23.809+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:23.810 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:23.809+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:24.513 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:24.513+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:24.513 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:24.513+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:24.513 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:24.513+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:24.514 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:24.513+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:24.514 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:24.513+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:24.519 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:24.517+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:24.519 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:24.517+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:24.519 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:24.517+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:24.520 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:24.517+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:24.564 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~131s 2020-05-18T22:12:24.564 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~131s 2020-05-18T22:12:24.565 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~131s 2020-05-18T22:12:24.565 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~131s 2020-05-18T22:12:24.565 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~131s 2020-05-18T22:12:24.565 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~131s 2020-05-18T22:12:24.566 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~131s 2020-05-18T22:12:24.566 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~131s 2020-05-18T22:12:24.667 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:24.665+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:24.668 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:24.665+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:24.668 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:24.665+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:24.668 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:24.665+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:24.846 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:24.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:24.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:24.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:24.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:24.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:24.847 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:24.845+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:25.474 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:25.468+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:25.474 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:25.468+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:25.474 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:25.468+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:25.474 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:25.468+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:25.498 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:25.492+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:25.498 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:25.496+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:25.499 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:25.496+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:25.499 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:25.496+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:25.499 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:25.496+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:25.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:25.616+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:25.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:25.616+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:25.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:25.616+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:25.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:25.616+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:25.819 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:25.816+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:25.819 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:25.816+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:25.820 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:25.816+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:25.820 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:25.816+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:26.518 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:26.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:26.518 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:26.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:26.519 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:26.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:26.519 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:26.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:26.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:26.532+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:26.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:26.532+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:26.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:26.532+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:26.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:26.532+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:26.536 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:26.532+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:26.589 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:26.584+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:26.590 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:26.584+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:26.590 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:26.584+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:26.591 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:26.584+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:26.780 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:26.776+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:26.780 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:26.776+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:26.781 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:26.776+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:26.781 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:26.776+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:27.521 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:27.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:27.522 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:27.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:27.522 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:27.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:27.522 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:27.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:27.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:27.528+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:27.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:27.528+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:27.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:27.528+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:27.530 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:27.528+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:27.530 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:27.528+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:27.594 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:27.588+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:27.595 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:27.588+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:27.595 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:27.588+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:27.595 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:27.588+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:27.758 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:27.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:27.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:27.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:27.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:27.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:27.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:27.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:28.480 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:28.476+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:28.481 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:28.476+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:28.481 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:28.476+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:28.481 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:28.476+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:28.482 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:28.476+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:28.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:28.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:28.498 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:28.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:28.499 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:28.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:28.499 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:28.496+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:28.569 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:12:28.564+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6434 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:12:28.616 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:28.612+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:28.617 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:28.612+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:28.617 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:28.612+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:28.617 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:28.612+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:28.788 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:28.784+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:28.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:28.784+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:28.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:28.784+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:28.789 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:28.784+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:29.508 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:29.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:29.509 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:29.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:29.509 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:29.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:29.509 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:29.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:29.511 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:29.508+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:29.511 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:29.508+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:29.512 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:29.508+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:29.512 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:29.508+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:29.512 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:29.508+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:29.583 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:29.580+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:29.584 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:29.580+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:29.584 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:29.580+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:29.584 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:29.580+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:29.777 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:29.776+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:29.777 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:29.776+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:29.777 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:29.776+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:29.778 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:29.776+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:30.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~137s 2020-05-18T22:12:30.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~137s 2020-05-18T22:12:30.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~137s 2020-05-18T22:12:30.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~137s 2020-05-18T22:12:30.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~137s 2020-05-18T22:12:30.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~137s 2020-05-18T22:12:30.274 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~137s 2020-05-18T22:12:30.274 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~137s 2020-05-18T22:12:30.508 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:30.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:30.509 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:30.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:30.509 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:30.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:30.510 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:30.504+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:30.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:30.556+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:30.560 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:30.556+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:30.560 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:30.556+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:30.560 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:30.556+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:30.560 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:30.556+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:30.582 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:30.580+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:30.582 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:30.580+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:30.582 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:30.580+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:30.583 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:30.580+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:30.771 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:30.768+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:30.771 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:30.768+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:30.772 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:30.768+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:30.772 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:30.768+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:31.494 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:31.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:31.494 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:31.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:31.495 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:31.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:31.495 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:31.488+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:31.542 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:31.540+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:31.543 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:31.540+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:31.543 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:31.540+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:31.543 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:31.540+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:31.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:31.548+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:31.553 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:31.548+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:31.553 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:31.548+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:31.553 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:31.548+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:31.554 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:31.548+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:31.722 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:31.720+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:31.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:31.720+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:31.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:31.720+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:31.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:31.720+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:32.516 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:32.512+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:32.516 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:32.512+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:32.517 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:32.512+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:32.517 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:32.512+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:32.522 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:32.516+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:32.522 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:32.516+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:32.522 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:32.516+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:32.523 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:32.516+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:32.543 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:32.540+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:32.543 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:32.540+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:32.543 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:32.540+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:32.544 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:32.540+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:32.544 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:32.540+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:32.707 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:32.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:32.708 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:32.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:32.708 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:32.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:32.708 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:32.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:33.520 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:33.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:33.521 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:33.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:33.521 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:33.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:33.521 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:33.516+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:33.529 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:33.524+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:33.529 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:33.524+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:33.529 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:33.524+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:33.529 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:33.524+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:33.571 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:12:33.568+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6470 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:12:33.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:33.580+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:33.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:33.580+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:33.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:33.580+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:33.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:33.580+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:33.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:33.580+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:33.694 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:33.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:33.694 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:33.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:33.695 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:33.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:33.695 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:33.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:34.553 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:34.552+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:34.553 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:34.552+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:34.553 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:34.552+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:34.553 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:34.552+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:34.559 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:34.556+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:34.559 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:34.556+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:34.560 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:34.556+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:34.560 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:34.556+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:34.577 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:34.572+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:34.577 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:34.572+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:34.578 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:34.572+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:34.578 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:34.572+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:34.578 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:34.572+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:34.698 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:34.692+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:34.698 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:34.692+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:34.698 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:34.692+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:34.699 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:34.692+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:35.547 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:35.544+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:35.548 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:35.544+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:35.548 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:35.544+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:35.548 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:35.544+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:35.598 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:35.596+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:35.598 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:35.596+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:35.598 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:35.596+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:35.599 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:35.596+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:35.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:35.612+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:35.616 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:35.612+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:35.616 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:35.612+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:35.616 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:35.612+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:35.616 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:35.612+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:35.745 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:35.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:35.745 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:35.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:35.745 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:35.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:35.746 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:35.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:35.981 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~143s 2020-05-18T22:12:35.981 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~143s 2020-05-18T22:12:35.982 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~143s 2020-05-18T22:12:35.982 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~143s 2020-05-18T22:12:35.982 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~143s 2020-05-18T22:12:35.982 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~143s 2020-05-18T22:12:35.982 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~143s 2020-05-18T22:12:35.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~143s 2020-05-18T22:12:36.559 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:36.556+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:36.560 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:36.556+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:36.560 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:36.556+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:36.560 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:36.556+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:36.567 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:36.564+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:36.567 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:36.564+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:36.568 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:36.564+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:36.568 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:36.564+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:36.632 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:36.628+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:36.632 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:36.628+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:36.632 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:36.628+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:36.632 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:36.628+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:36.633 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:36.628+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:36.710 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:36.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:36.710 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:36.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:36.711 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:36.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:36.711 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:36.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:37.512 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:37.508+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:37.513 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:37.508+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:37.513 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:37.508+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:37.513 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:37.508+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:37.605 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:37.600+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:37.605 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:37.600+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:37.606 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:37.600+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:37.606 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:37.600+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:37.620 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:37.616+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:37.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:37.616+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:37.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:37.616+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:37.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:37.616+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:37.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:37.616+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:37.742 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:37.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:37.743 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:37.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:37.743 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:37.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:37.743 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:37.740+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:38.555 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:38.552+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:38.556 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:38.552+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:38.556 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:38.552+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:38.556 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:38.552+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:38.573 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:12:38.568+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6506 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:12:38.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:38.584+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:38.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:38.584+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:38.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:38.584+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:38.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:38.584+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:38.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:38.584+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:38.614 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:38.612+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:38.615 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:38.612+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:38.615 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:38.612+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:38.615 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:38.612+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:38.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:38.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:38.707 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:38.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:38.707 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:38.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:38.707 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:38.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:39.584 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:39.580+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:39.584 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:39.580+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:39.584 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:39.580+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:39.585 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:39.580+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:39.585 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:39.580+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:39.602 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:39.600+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:39.602 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:39.600+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:39.602 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:39.600+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:39.603 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:39.600+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:39.629 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:39.624+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:39.630 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:39.624+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:39.630 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:39.624+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:39.631 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:39.624+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:39.734 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:39.732+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:39.735 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:39.732+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:39.735 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:39.732+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:39.735 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:39.732+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:40.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:40.532+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:40.536 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:40.532+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:40.536 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:40.532+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:40.536 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:40.532+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:40.536 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:40.532+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:40.604 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:40.604+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:40.605 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:40.604+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:40.605 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:40.604+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:40.605 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:40.604+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:40.619 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:40.616+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:40.619 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:40.616+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:40.619 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:40.616+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:40.619 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:40.616+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:40.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:40.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:40.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:40.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:40.748 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:40.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:40.748 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:40.744+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:41.516 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:41.512+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:41.516 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:41.512+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:41.516 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:41.512+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:41.517 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:41.512+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:41.517 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:41.512+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:41.597 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:41.592+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:41.597 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:41.592+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:41.598 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:41.592+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:41.598 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:41.592+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:41.612 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:41.608+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:41.612 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:41.608+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:41.612 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:41.608+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:41.613 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:41.608+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:41.689 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~148s 2020-05-18T22:12:41.690 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~148s 2020-05-18T22:12:41.690 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~148s 2020-05-18T22:12:41.690 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~148s 2020-05-18T22:12:41.690 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~148s 2020-05-18T22:12:41.690 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~148s 2020-05-18T22:12:41.691 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~148s 2020-05-18T22:12:41.691 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~148s 2020-05-18T22:12:41.732 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:41.728+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:41.733 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:41.728+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:41.733 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:41.728+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:41.733 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:41.728+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:42.558 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:42.556+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:42.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:42.556+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:42.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:42.556+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:42.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:42.556+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:42.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:42.556+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:42.616 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:42.612+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:42.616 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:42.612+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:42.616 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:42.612+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:42.617 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:42.612+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:42.661 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:42.656+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:42.661 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:42.656+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:42.661 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:42.656+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:42.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:42.656+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:42.760 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:42.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:42.760 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:42.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:42.760 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:42.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:42.760 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:42.756+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:43.574 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:12:43.572+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6542 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:12:43.580 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:43.576+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:43.580 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:43.576+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:43.581 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:43.576+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:43.581 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:43.576+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:43.598 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:43.592+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:43.598 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:43.592+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:43.598 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:43.592+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:43.598 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:43.592+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:43.599 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:43.596+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:43.626 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:43.624+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:43.627 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:43.624+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:43.627 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:43.624+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:43.627 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:43.624+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:43.783 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:43.780+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:43.783 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:43.780+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:43.784 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:43.780+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:43.784 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:43.780+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:44.582 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:44.580+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:44.583 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:44.580+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:44.583 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:44.580+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:44.583 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:44.580+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:44.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:44.584+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:44.589 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:44.584+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:44.589 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:44.584+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:44.589 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:44.584+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:44.590 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:44.584+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:44.662 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:44.660+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:44.663 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:44.660+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:44.663 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:44.660+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:44.663 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:44.660+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:44.739 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:44.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:44.740 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:44.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:44.740 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:44.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:44.740 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:44.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:45.587 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:45.584+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:45.587 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:45.584+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:45.587 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:45.584+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:45.588 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:45.584+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:45.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:45.592+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:45.596 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:45.592+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:45.596 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:45.592+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:45.596 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:45.592+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:45.596 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:45.592+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:45.704 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:45.700+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:45.705 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:45.700+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:45.705 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:45.700+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:45.705 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:45.700+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:45.718 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:45.712+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:45.718 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:45.712+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:45.718 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:45.712+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:45.718 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:45.712+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:46.584 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:46.580+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:46.585 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:46.580+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:46.585 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:46.580+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:46.585 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:46.580+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:46.617 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:46.616+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:46.617 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:46.616+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:46.618 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:46.616+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:46.618 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:46.616+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:46.618 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:46.616+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:46.726 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:46.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:46.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:46.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:46.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:46.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:46.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:46.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:46.749 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:46.744+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:46.749 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:46.744+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:46.749 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:46.744+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:46.750 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:46.744+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:47.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~154s 2020-05-18T22:12:47.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~154s 2020-05-18T22:12:47.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~154s 2020-05-18T22:12:47.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~154s 2020-05-18T22:12:47.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~154s 2020-05-18T22:12:47.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~154s 2020-05-18T22:12:47.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~154s 2020-05-18T22:12:47.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~154s 2020-05-18T22:12:47.552 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:47.548+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:47.552 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:47.548+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:47.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:47.548+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:47.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:47.548+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:47.625 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:47.624+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:47.626 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:47.624+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:47.626 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:47.624+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:47.626 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:47.624+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:47.626 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:47.624+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:47.768 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:47.764+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:47.768 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:47.764+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:47.769 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:47.764+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:47.769 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:47.764+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:47.787 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:47.784+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:47.787 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:47.784+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:47.787 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:47.784+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:47.788 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:47.784+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:48.576 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:12:48.572+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6577 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:12:48.600 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:48.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:48.601 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:48.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:48.601 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:48.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:48.601 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:48.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:48.625 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:48.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:48.625 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:48.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:48.625 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:48.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:48.625 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:48.620+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:48.626 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:48.620+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:48.731 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:48.728+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:48.732 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:48.728+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:48.732 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:48.728+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:48.732 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:48.728+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:48.766 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:48.760+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:48.766 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:48.760+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:48.774 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:48.760+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:48.774 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:48.760+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:49.552 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:49.552+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:49.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:49.552+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:49.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:49.552+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:49.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:49.552+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:49.673 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:49.672+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:49.673 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:49.672+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:49.673 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:49.672+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:49.674 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:49.672+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:49.674 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:49.672+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:49.686 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:49.684+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:49.686 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:49.684+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:49.687 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:49.684+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:49.687 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:49.684+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:49.785 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:49.784+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:49.785 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:49.784+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:49.786 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:49.784+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:49.786 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:49.784+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:50.552 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:50.548+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:50.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:50.548+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:50.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:50.548+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:50.553 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:50.548+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:50.659 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:50.656+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:50.660 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:50.656+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:50.660 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:50.656+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:50.660 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:50.656+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:50.660 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:50.656+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:50.719 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:50.716+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:50.719 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:50.716+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:50.719 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:50.716+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:50.719 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:50.716+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:50.832 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:50.832+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:50.833 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:50.832+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:50.833 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:50.832+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:50.833 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:50.832+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:51.512 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:51.508+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:51.512 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:51.508+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:51.513 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:51.508+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:51.513 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:51.508+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:51.691 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:51.688+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:51.692 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:51.688+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:51.692 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:51.688+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:51.692 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:51.688+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:51.692 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:51.688+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:51.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:12:51.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:12:51.736 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:51.732+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:51.736 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:51.732+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:51.736 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:51.732+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:51.737 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:51.732+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:51.818 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:51.816+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:51.818 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:51.816+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:51.819 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:51.816+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:51.819 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:51.816+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:52.524 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:52.520+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:52.525 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:52.520+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:52.525 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:52.520+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:52.525 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:52.520+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:52.713 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:52.708+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:52.713 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:52.708+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:52.713 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:52.708+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:52.714 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:52.708+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:52.714 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:52.708+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:52.738 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:52.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:52.739 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:52.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:52.739 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:52.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:52.739 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:52.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:52.783 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:52.780+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:52.784 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:52.780+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:52.784 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:52.780+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:52.784 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:52.780+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:53.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~160s 2020-05-18T22:12:53.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~160s 2020-05-18T22:12:53.106 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~160s 2020-05-18T22:12:53.106 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~160s 2020-05-18T22:12:53.106 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~160s 2020-05-18T22:12:53.106 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~160s 2020-05-18T22:12:53.107 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~160s 2020-05-18T22:12:53.107 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~160s 2020-05-18T22:12:53.564 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:53.560+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:53.565 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:53.560+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:53.565 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:53.560+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:53.565 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:53.560+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:53.581 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:12:53.572+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6613 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:12:53.699 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:53.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:53.700 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:53.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:53.700 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:53.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:53.700 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:53.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:53.718 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:53.716+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:53.718 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:53.716+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:53.719 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:53.716+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:53.719 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:53.716+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:53.719 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:53.716+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:53.754 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:53.748+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:53.754 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:53.748+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:53.754 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:53.748+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:53.755 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:53.748+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:54.609 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:54.604+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:54.609 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:54.604+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:54.610 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:54.604+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:54.610 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:54.604+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:54.686 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:54.684+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:54.687 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:54.684+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:54.687 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:54.684+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:54.687 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:54.684+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:54.687 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:54.684+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:54.705 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:54.700+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:54.705 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:54.700+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:54.706 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:54.700+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:54.706 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:54.700+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:54.711 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:54.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:54.711 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:54.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:54.711 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:54.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:54.712 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:54.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:54.782 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:12:54.782 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:12:55.599 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:55.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:55.600 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:55.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:55.600 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:55.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:55.600 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:55.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:55.643 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:55.640+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:55.644 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:55.640+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:55.644 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:55.640+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:55.644 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:55.640+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:55.644 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:55.640+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:55.666 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:55.664+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:55.666 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:55.664+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:55.667 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:55.664+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:55.667 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:55.664+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:55.739 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:55.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:55.739 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:55.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:55.739 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:55.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:55.739 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:55.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:56.573 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:56.568+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:56.574 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:56.568+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:56.574 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:56.568+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:56.574 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:56.568+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:56.612 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:56.608+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:56.613 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:56.608+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:56.613 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:56.608+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:56.613 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:56.608+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:56.614 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:56.608+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:56.684 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:56.684+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:56.685 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:56.684+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:56.685 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:56.684+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:56.685 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:56.684+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:56.712 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:56.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:56.713 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:56.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:56.713 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:56.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:56.713 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:56.708+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:57.547 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:57.544+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:57.548 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:57.544+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:57.548 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:57.544+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:57.548 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:57.544+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:57.603 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:57.600+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:57.603 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:57.600+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:57.604 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:57.600+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:57.604 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:57.600+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:57.605 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:57.600+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:57.639 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:57.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:57.640 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:57.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:57.640 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:57.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:57.640 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:57.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:57.687 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:57.684+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:57.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:57.684+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:57.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:57.684+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:57.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:57.684+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:58.530 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:58.528+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:58.530 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:58.528+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:58.531 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:58.528+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:58.531 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:58.528+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:58.561 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:58.560+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:58.562 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:58.560+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:58.562 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:58.560+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:58.562 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:58.560+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:58.562 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:58.560+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:58.579 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:12:58.576+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6649 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:12:58.617 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:58.612+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:58.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:58.612+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:58.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:58.612+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:58.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:58.612+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:58.675 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:58.672+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:58.675 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:58.672+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:58.675 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:58.672+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:58.675 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:58.672+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:58.813 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~166s 2020-05-18T22:12:58.813 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~166s 2020-05-18T22:12:58.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~166s 2020-05-18T22:12:58.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~166s 2020-05-18T22:12:58.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~166s 2020-05-18T22:12:58.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~166s 2020-05-18T22:12:58.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~166s 2020-05-18T22:12:58.815 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~166s 2020-05-18T22:12:59.502 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:59.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:59.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:59.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:59.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:59.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:59.503 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:12:59.500+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:12:59.566 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:59.564+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:59.567 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:59.564+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:59.567 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:59.564+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:59.567 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:59.564+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:12:59.567 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:12:59.564+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:12:59.576 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:59.576+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:59.577 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:59.576+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:59.577 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:59.576+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:59.577 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:12:59.576+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:12:59.658 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:59.656+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:59.658 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:59.656+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:59.658 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:59.656+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:12:59.658 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:12:59.656+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:00.550 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:00.548+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:00.550 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:00.548+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:00.550 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:00.548+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:00.550 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:00.548+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:00.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:00.560+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:00.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:00.560+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:00.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:00.560+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:00.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:00.560+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:00.564 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:00.560+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:00.616 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:00.616+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:00.617 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:00.616+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:00.617 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:00.616+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:00.617 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:00.616+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:00.665 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:00.660+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:00.665 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:00.660+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:00.666 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:00.660+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:00.666 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:00.660+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:01.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:01.584+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:01.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:01.584+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:01.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:01.584+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:01.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:01.584+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:01.589 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:01.584+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:01.599 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:01.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:01.600 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:01.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:01.600 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:01.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:01.600 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:01.596+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:01.623 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:01.620+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:01.623 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:01.620+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:01.623 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:01.620+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:01.624 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:01.620+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:01.624 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:01.620+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:01.624 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:01.620+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:01.625 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:01.620+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:01.625 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:01.620+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:02.561 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:02.556+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:02.561 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:02.556+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:02.561 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:02.556+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:02.562 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:02.556+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:02.562 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:02.556+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:02.648 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:02.644+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:02.648 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:02.644+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:02.649 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:02.644+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:02.649 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:02.644+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:02.666 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:02.664+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:02.666 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:02.664+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:02.667 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:02.664+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:02.667 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:02.664+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:02.671 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:02.668+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:02.671 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:02.668+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:02.671 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:02.668+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:02.671 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:02.668+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:03.571 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:03.568+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:03.571 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:03.568+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:03.571 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:03.568+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:03.572 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:03.568+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:03.572 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:03.568+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:03.582 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:13:03.576+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6685 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:13:03.635 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:03.632+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:03.635 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:03.632+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:03.635 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:03.632+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:03.636 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:03.632+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:03.697 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:03.696+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:03.698 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:03.696+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:03.698 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:03.696+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:03.698 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:03.696+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:03.715 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:03.712+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:03.715 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:03.712+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:03.715 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:03.712+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:03.716 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:03.712+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:04.520 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~171s 2020-05-18T22:13:04.521 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~171s 2020-05-18T22:13:04.521 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~171s 2020-05-18T22:13:04.521 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~171s 2020-05-18T22:13:04.522 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~171s 2020-05-18T22:13:04.522 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~171s 2020-05-18T22:13:04.522 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~171s 2020-05-18T22:13:04.522 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~171s 2020-05-18T22:13:04.598 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:04.596+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:04.599 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:04.596+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:04.599 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:04.596+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:04.599 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:04.596+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:04.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:04.608+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:04.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:04.608+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:04.609 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:04.608+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:04.609 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:04.608+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:04.609 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:04.608+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:04.705 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:04.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:04.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:04.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:04.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:04.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:04.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:04.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:04.729 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:04.724+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:04.729 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:04.724+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:04.729 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:04.724+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:04.730 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:04.724+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:05.607 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:05.600+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:05.608 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:05.600+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:05.608 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:05.600+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:05.608 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:05.600+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:05.654 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:05.652+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:05.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:05.652+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:05.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:05.652+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:05.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:05.652+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:05.656 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:05.652+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:05.694 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:05.688+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:05.694 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:05.688+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:05.694 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:05.688+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:05.695 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:05.688+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:05.695 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:05.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:05.695 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:05.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:05.695 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:05.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:05.695 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:05.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:06.640 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:06.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:06.641 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:06.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:06.641 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:06.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:06.641 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:06.636+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:06.657 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:06.652+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:06.657 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:06.652+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:06.658 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:06.652+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:06.658 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:06.652+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:06.691 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:06.688+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:06.691 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:06.688+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:06.691 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:06.688+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:06.691 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:06.688+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:06.692 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:06.688+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:06.708 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:06.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:06.709 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:06.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:06.709 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:06.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:06.709 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:06.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:07.616 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:07.612+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:07.616 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:07.612+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:07.616 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:07.612+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:07.617 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:07.612+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:07.671 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:07.668+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:07.672 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:07.668+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:07.672 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:07.668+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:07.672 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:07.668+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:07.673 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:07.668+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:07.690 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:07.688+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:07.691 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:07.688+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:07.691 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:07.688+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:07.691 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:07.688+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:07.726 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:07.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:07.726 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:07.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:07.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:07.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:07.727 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:07.724+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:08.583 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:13:08.580+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6720 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:13:08.604 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:08.600+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:08.605 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:08.600+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:08.605 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:08.600+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:08.605 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:08.600+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:08.630 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:08.624+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:08.630 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:08.624+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:08.630 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:08.624+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:08.631 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:08.624+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:08.631 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:08.628+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:08.684 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:08.680+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:08.684 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:08.680+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:08.684 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:08.680+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:08.684 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:08.680+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:08.709 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:08.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:08.709 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:08.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:08.710 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:08.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:08.710 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:08.704+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:09.621 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:09.620+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:09.621 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:09.620+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:09.621 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:09.620+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:09.622 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:09.620+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:09.649 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:09.648+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:09.649 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:09.648+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:09.649 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:09.648+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:09.650 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:09.648+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:09.650 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:09.648+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:09.700 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:09.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:09.701 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:09.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:09.701 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:09.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:09.701 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:09.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:09.718 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:09.716+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:09.719 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:09.716+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:09.719 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:09.716+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:09.719 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:09.716+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:10.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~177s 2020-05-18T22:13:10.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~177s 2020-05-18T22:13:10.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~177s 2020-05-18T22:13:10.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~177s 2020-05-18T22:13:10.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~177s 2020-05-18T22:13:10.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~177s 2020-05-18T22:13:10.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~177s 2020-05-18T22:13:10.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~177s 2020-05-18T22:13:10.617 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:10.616+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:10.617 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:10.616+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:10.617 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:10.616+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:10.618 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:10.616+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:10.618 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:10.616+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:10.656 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:10.656+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:10.656 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:10.656+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:10.657 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:10.656+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:10.657 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:10.656+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:10.738 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:10.736+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:10.738 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:10.736+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:10.739 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:10.736+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:10.739 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:10.736+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:10.741 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:10.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:10.741 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:10.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:10.742 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:10.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:10.742 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:10.736+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:11.630 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:11.628+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:11.631 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:11.628+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:11.631 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:11.628+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:11.631 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:11.628+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:11.632 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:11.628+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:11.659 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:11.652+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:11.660 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:11.652+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:11.660 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:11.652+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:11.660 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:11.652+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:11.700 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:11.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:11.700 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:11.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:11.700 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:11.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:11.700 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:11.696+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:11.745 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:11.744+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:11.746 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:11.744+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:11.746 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:11.744+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:11.746 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:11.744+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:12.666 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:12.664+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:12.666 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:12.664+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:12.667 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:12.664+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:12.667 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:12.664+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:12.667 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:12.664+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:12.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:12.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:12.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:12.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:12.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:12.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:12.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:12.688+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:12.702 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:12.700+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:12.702 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:12.700+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:12.703 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:12.700+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:12.703 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:12.700+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:12.703 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:12.700+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:12.704 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:12.700+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:12.704 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:12.700+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:12.704 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:12.700+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:13.585 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:13:13.579+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6756 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:13:13.626 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:13.623+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:13.626 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:13.623+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:13.627 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:13.623+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:13.627 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:13.623+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:13.627 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:13.623+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:13.678 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:13.675+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:13.678 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:13.675+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:13.678 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:13.675+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:13.678 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:13.675+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:13.714 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:13.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:13.714 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:13.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:13.714 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:13.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:13.715 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:13.707+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:13.746 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:13.743+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:13.747 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:13.743+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:13.747 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:13.743+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:13.747 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:13.743+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:14.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:14.603+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:14.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:14.603+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:14.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:14.603+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:14.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:14.603+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:14.609 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:14.603+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:14.691 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:14.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:14.691 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:14.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:14.691 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:14.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:14.691 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:14.687+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:14.705 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:14.699+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:14.705 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:14.699+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:14.705 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:14.699+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:14.705 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:14.699+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:14.756 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:14.755+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:14.756 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:14.755+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:14.756 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:14.755+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:14.757 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:14.755+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:15.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:15.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:15.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:15.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:15.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:15.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:15.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:15.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:15.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:15.583+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:15.650 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:15.647+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:15.650 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:15.647+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:15.650 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:15.647+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:15.651 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:15.647+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:15.687 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:15.683+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:15.687 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:15.683+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:15.687 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:15.683+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:15.688 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:15.683+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:15.758 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:15.755+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:15.758 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:15.755+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:15.759 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:15.755+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:15.759 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:15.755+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:15.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~183s 2020-05-18T22:13:15.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~183s 2020-05-18T22:13:15.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~183s 2020-05-18T22:13:15.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~183s 2020-05-18T22:13:15.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~183s 2020-05-18T22:13:15.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~183s 2020-05-18T22:13:15.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~183s 2020-05-18T22:13:15.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~183s 2020-05-18T22:13:16.599 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:16.595+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:16.599 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:16.595+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:16.600 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:16.595+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:16.600 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:16.595+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:16.600 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:16.595+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:16.626 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:16.623+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:16.626 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:16.623+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:16.626 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:16.623+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:16.627 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:16.623+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:16.719 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:16.715+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:16.719 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:16.715+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:16.719 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:16.715+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:16.720 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:16.715+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:16.721 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:16.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:16.722 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:16.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:16.722 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:16.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:16.722 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:16.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:17.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:17.611+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:17.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:17.611+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:17.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:17.611+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:17.615 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:17.611+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:17.616 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:17.611+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:17.667 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:17.667+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:17.668 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:17.667+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:17.668 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:17.667+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:17.668 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:17.667+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:17.683 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:17.679+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:17.684 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:17.679+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:17.684 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:17.679+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:17.685 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:17.679+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:17.700 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:17.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:17.700 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:17.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:17.701 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:17.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:17.701 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:17.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:18.587 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:13:18.583+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6792 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:13:18.621 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:18.615+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:18.621 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:18.615+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:18.622 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:18.615+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:18.622 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:18.615+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:18.662 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:18.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:18.663 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:18.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:18.663 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:18.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:18.663 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:18.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:18.663 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:18.659+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:18.684 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:18.679+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:18.686 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:18.679+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:18.686 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:18.679+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:18.686 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:18.679+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:18.719 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:18.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:18.719 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:18.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:18.720 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:18.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:18.720 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:18.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:19.599 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:19.595+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:19.599 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:19.595+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:19.600 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:19.595+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:19.600 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:19.595+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:19.670 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:19.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:19.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:19.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:19.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:19.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:19.671 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:19.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:19.671 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:19.663+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:19.672 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:19.663+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:19.672 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:19.663+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:19.672 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:19.663+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:19.672 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:19.663+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:19.690 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:19.687+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:19.691 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:19.687+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:19.691 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:19.687+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:19.691 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:19.687+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:20.586 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:20.583+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:20.587 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:20.583+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:20.587 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:20.583+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:20.587 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:20.583+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:20.634 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:20.627+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:20.635 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:20.627+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:20.635 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:20.627+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:20.635 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:20.627+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:20.648 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:20.643+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:20.649 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:20.643+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:20.649 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:20.643+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:20.649 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:20.643+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:20.649 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:20.643+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:20.712 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:20.711+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:20.713 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:20.711+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:20.713 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:20.711+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:20.713 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:20.711+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:21.594 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:21.591+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:21.595 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:21.591+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:21.595 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:21.591+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:21.595 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:21.591+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:21.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:21.619+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:21.620 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:21.619+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:21.620 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:21.619+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:21.620 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:21.619+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:21.644 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~188s 2020-05-18T22:13:21.644 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~188s 2020-05-18T22:13:21.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~188s 2020-05-18T22:13:21.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~188s 2020-05-18T22:13:21.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~188s 2020-05-18T22:13:21.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~188s 2020-05-18T22:13:21.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~188s 2020-05-18T22:13:21.646 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~188s 2020-05-18T22:13:21.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:21.651+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:21.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:21.651+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:21.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:21.651+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:21.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:21.651+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:21.655 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:21.651+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:21.718 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:21.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:21.719 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:21.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:21.719 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:21.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:21.719 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:21.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:22.605 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:22.599+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:22.606 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:22.599+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:22.606 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:22.599+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:22.606 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:22.599+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:22.644 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:22.643+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:22.644 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:22.643+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:22.644 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:22.643+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:22.645 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:22.643+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:22.663 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:22.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:22.664 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:22.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:22.664 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:22.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:22.665 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:22.659+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:22.665 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:22.659+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:22.675 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:22.671+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:22.675 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:22.671+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:22.675 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:22.671+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:22.676 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:22.671+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:23.588 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:13:23.583+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6828 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:13:23.630 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:23.623+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:23.630 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:23.623+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:23.630 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:23.623+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:23.630 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:23.623+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:23.641 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:23.635+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:23.641 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:23.635+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:23.642 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:23.635+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:23.642 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:23.635+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:23.642 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:23.635+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:23.652 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:23.647+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:23.652 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:23.647+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:23.652 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:23.647+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:23.652 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:23.647+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:23.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:23.683+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:23.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:23.683+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:23.688 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:23.683+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:23.689 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:23.683+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:24.586 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:24.583+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:24.586 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:24.583+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:24.587 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:24.583+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:24.587 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:24.583+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:24.608 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:24.603+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:24.608 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:24.603+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:24.608 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:24.603+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:24.609 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:24.603+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:24.627 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:24.623+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:24.628 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:24.623+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:24.628 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:24.623+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:24.628 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:24.623+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:24.628 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:24.623+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:24.658 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:24.655+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:24.659 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:24.655+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:24.659 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:24.655+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:24.659 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:24.655+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:24.784 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:13:24.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:13:25.561 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:25.559+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:25.562 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:25.559+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:25.562 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:25.559+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:25.562 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:25.559+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:25.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:25.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:25.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:25.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:25.589 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:25.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:25.589 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:25.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:25.589 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:25.583+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:25.590 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:25.587+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:25.590 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:25.587+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:25.591 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:25.587+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:25.591 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:25.587+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:25.612 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:25.611+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:25.612 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:25.611+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:25.613 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:25.611+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:25.613 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:25.611+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:26.562 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:26.559+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:26.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:26.559+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:26.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:26.559+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:26.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:26.559+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:26.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:26.559+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:26.586 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:26.583+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:26.586 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:26.583+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:26.587 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:26.583+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:26.587 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:26.583+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:26.608 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:26.603+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:26.608 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:26.603+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:26.609 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:26.603+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:26.609 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:26.603+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:26.617 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:26.615+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:26.618 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:26.615+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:26.618 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:26.615+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:26.618 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:26.615+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:27.351 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~194s 2020-05-18T22:13:27.352 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~194s 2020-05-18T22:13:27.352 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~194s 2020-05-18T22:13:27.352 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~194s 2020-05-18T22:13:27.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~194s 2020-05-18T22:13:27.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~194s 2020-05-18T22:13:27.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~194s 2020-05-18T22:13:27.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~194s 2020-05-18T22:13:27.569 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:27.567+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:27.569 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:27.567+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:27.569 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:27.567+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:27.570 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:27.567+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:27.570 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:27.567+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:27.575 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:27.571+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:27.575 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:27.571+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:27.575 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:27.571+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:27.576 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:27.571+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:27.612 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:27.607+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:27.613 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:27.607+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:27.613 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:27.607+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:27.613 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:27.607+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:27.657 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:27.651+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:27.657 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:27.651+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:27.658 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:27.651+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:27.658 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:27.651+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:27.842 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:13:27.842 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:13:28.590 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:13:28.587+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6863 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:13:28.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:28.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:28.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:28.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:28.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:28.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:28.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:28.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:28.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:28.591+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:28.606 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:28.603+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:28.606 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:28.603+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:28.607 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:28.603+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:28.607 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:28.603+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:28.611 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:28.607+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:28.611 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:28.607+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:28.612 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:28.607+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:28.612 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:28.607+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:28.621 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:28.615+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:28.621 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:28.615+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:28.621 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:28.615+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:28.621 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:28.615+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:29.585 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:29.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:29.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:29.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:29.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:29.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:29.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:29.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:29.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:29.583+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:29.587 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:29.583+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:29.587 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:29.583+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:29.587 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:29.583+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:29.587 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:29.583+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:29.601 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:29.595+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:29.601 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:29.595+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:29.602 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:29.595+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:29.602 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:29.595+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:29.625 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:29.623+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:29.626 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:29.623+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:29.627 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:29.623+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:29.627 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:29.623+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:30.595 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:30.591+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:30.596 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:30.591+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:30.596 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:30.591+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:30.596 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:30.591+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:30.623 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:30.619+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:30.624 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:30.619+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:30.624 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:30.619+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:30.624 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:30.619+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:30.624 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:30.619+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:30.626 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:30.623+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:30.626 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:30.623+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:30.626 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:30.623+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:30.626 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:30.623+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:30.653 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:30.651+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:30.653 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:30.651+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:30.653 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:30.651+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:30.654 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:30.651+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:31.560 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:31.555+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:31.561 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:31.555+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:31.561 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:31.555+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:31.561 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:31.555+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:31.579 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:31.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:31.580 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:31.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:31.580 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:31.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:31.580 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:31.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:31.580 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:31.575+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:31.622 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:31.619+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:31.623 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:31.619+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:31.623 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:31.619+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:31.623 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:31.619+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:31.666 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:31.663+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:31.666 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:31.663+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:31.667 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:31.663+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:31.667 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:31.663+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:32.592 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:32.587+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:32.592 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:32.587+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:32.592 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:32.587+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:32.593 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:32.587+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:32.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:32.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:32.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:32.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:32.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:32.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:32.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:32.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:32.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:32.591+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:32.648 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:32.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:32.648 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:32.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:32.649 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:32.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:32.649 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:32.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:32.693 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:32.691+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:32.694 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:32.691+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:32.694 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:32.691+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:32.694 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:32.691+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:33.059 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~200s 2020-05-18T22:13:33.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~200s 2020-05-18T22:13:33.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~200s 2020-05-18T22:13:33.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~200s 2020-05-18T22:13:33.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~200s 2020-05-18T22:13:33.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~200s 2020-05-18T22:13:33.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~200s 2020-05-18T22:13:33.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~200s 2020-05-18T22:13:33.592 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:13:33.587+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6899 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:13:33.607 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:33.595+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:33.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:33.595+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:33.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:33.595+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:33.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:33.595+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:33.608 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:33.595+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:33.622 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:33.619+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:33.622 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:33.619+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:33.622 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:33.619+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:33.623 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:33.619+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:33.656 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:33.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:33.656 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:33.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:33.656 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:33.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:33.657 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:33.655+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:33.701 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:33.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:33.702 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:33.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:33.702 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:33.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:33.702 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:33.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:34.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:34.555+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:34.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:34.555+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:34.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:34.555+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:34.560 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:34.555+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:34.560 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:34.555+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:34.651 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:34.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:34.651 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:34.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:34.651 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:34.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:34.652 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:34.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:34.672 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:34.667+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:34.672 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:34.667+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:34.672 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:34.667+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:34.672 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:34.667+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:34.690 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:34.683+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:34.690 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:34.683+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:34.690 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:34.683+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:34.690 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:34.683+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:35.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:35.523+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:35.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:35.523+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:35.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:35.523+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:35.530 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:35.523+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:35.530 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:35.523+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:35.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:35.643+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:35.645 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:35.643+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:35.645 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:35.643+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:35.645 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:35.643+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:35.683 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:35.679+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:35.683 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:35.679+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:35.684 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:35.679+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:35.684 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:35.679+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:35.705 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:35.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:35.705 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:35.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:35.705 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:35.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:35.706 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:35.703+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:36.579 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:36.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:36.579 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:36.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:36.579 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:36.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:36.580 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:36.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:36.580 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:36.575+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:36.664 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:36.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:36.665 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:36.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:36.665 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:36.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:36.665 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:36.659+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:36.715 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:36.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:36.715 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:36.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:36.716 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:36.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:36.716 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:36.715+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:36.719 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:36.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:36.719 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:36.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:36.719 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:36.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:36.720 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:36.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:37.599 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:37.595+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:37.599 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:37.595+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:37.600 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:37.595+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:37.600 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:37.595+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:37.600 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:37.595+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:37.650 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:37.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:37.650 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:37.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:37.651 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:37.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:37.651 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:37.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:37.695 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:37.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:37.696 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:37.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:37.696 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:37.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:37.696 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:37.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:37.704 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:37.699+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:37.704 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:37.699+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:37.705 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:37.699+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:37.705 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:37.699+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:38.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:38.551+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:38.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:38.551+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:38.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:38.551+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:38.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:38.551+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:38.553 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:38.551+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:38.593 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:13:38.591+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6935 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:13:38.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:38.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:38.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:38.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:38.645 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:38.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:38.645 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:38.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:38.699 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:38.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:38.699 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:38.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:38.699 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:38.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:38.700 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:38.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:38.717 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:38.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:38.717 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:38.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:38.717 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:38.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:38.717 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:38.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:38.766 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~205s 2020-05-18T22:13:38.766 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~205s 2020-05-18T22:13:38.767 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~205s 2020-05-18T22:13:38.767 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~205s 2020-05-18T22:13:38.767 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~205s 2020-05-18T22:13:38.767 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~205s 2020-05-18T22:13:38.767 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~205s 2020-05-18T22:13:38.768 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~205s 2020-05-18T22:13:39.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:39.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:39.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:39.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:39.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:39.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:39.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:39.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:39.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:39.583+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:39.674 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:39.671+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:39.674 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:39.671+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:39.675 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:39.671+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:39.675 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:39.671+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:39.680 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:39.675+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:39.681 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:39.675+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:39.681 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:39.675+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:39.681 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:39.675+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:39.725 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:39.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:39.725 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:39.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:39.725 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:39.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:39.725 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:39.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:40.612 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:40.607+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:40.613 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:40.607+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:40.613 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:40.607+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:40.613 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:40.607+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:40.613 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:40.607+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:40.639 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:40.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:40.639 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:40.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:40.639 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:40.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:40.640 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:40.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:40.674 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:40.671+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:40.674 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:40.671+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:40.675 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:40.671+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:40.675 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:40.671+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:40.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:40.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:40.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:40.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:40.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:40.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:40.775 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:40.767+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:41.565 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:41.559+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:41.566 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:41.559+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:41.566 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:41.559+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:41.566 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:41.559+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:41.566 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:41.559+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:41.650 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:41.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:41.650 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:41.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:41.650 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:41.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:41.650 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:41.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:41.724 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:41.723+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:41.724 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:41.723+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:41.725 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:41.723+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:41.725 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:41.723+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:41.773 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:41.771+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:41.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:41.771+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:41.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:41.771+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:41.774 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:41.771+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:42.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:42.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:42.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:42.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:42.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:42.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:42.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:42.583+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:42.588 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:42.583+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:42.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:42.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:42.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:42.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:42.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:42.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:42.644 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:42.639+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:42.732 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:42.731+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:42.733 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:42.731+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:42.733 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:42.731+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:42.733 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:42.731+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:42.762 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:42.759+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:42.763 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:42.759+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:42.763 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:42.759+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:42.763 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:42.759+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:43.595 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:13:43.591+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 6971 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:13:43.629 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:43.623+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:43.629 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:43.623+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:43.630 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:43.623+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:43.630 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:43.623+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:43.630 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:43.623+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:43.654 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:43.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:43.654 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:43.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:43.654 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:43.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:43.654 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:43.647+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:43.716 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:43.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:43.716 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:43.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:43.717 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:43.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:43.717 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:43.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:43.741 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:43.739+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:43.741 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:43.739+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:43.741 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:43.739+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:43.742 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:43.739+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:44.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~211s 2020-05-18T22:13:44.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~211s 2020-05-18T22:13:44.475 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~211s 2020-05-18T22:13:44.475 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~211s 2020-05-18T22:13:44.475 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~211s 2020-05-18T22:13:44.475 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~211s 2020-05-18T22:13:44.475 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~211s 2020-05-18T22:13:44.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~211s 2020-05-18T22:13:44.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:44.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:44.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:44.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:44.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:44.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:44.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:44.591+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:44.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:44.591+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:44.618 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:44.615+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:44.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:44.615+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:44.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:44.615+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:44.619 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:44.615+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:44.671 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:44.671+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:44.672 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:44.671+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:44.672 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:44.671+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:44.672 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:44.671+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:44.780 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:44.775+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:44.781 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:44.775+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:44.781 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:44.775+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:44.781 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:44.775+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:45.545 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:45.543+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:45.545 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:45.543+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:45.545 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:45.543+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:45.546 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:45.543+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:45.546 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:45.543+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:45.594 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:45.591+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:45.594 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:45.591+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:45.594 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:45.591+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:45.595 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:45.591+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:45.694 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:45.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:45.695 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:45.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:45.695 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:45.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:45.695 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:45.691+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:45.794 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:45.791+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:45.795 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:45.791+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:45.795 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:45.791+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:45.795 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:45.791+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:46.560 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:46.555+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:46.560 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:46.555+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:46.561 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:46.555+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:46.561 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:46.555+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:46.561 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:46.555+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:46.601 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:46.599+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:46.602 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:46.599+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:46.602 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:46.599+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:46.602 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:46.599+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:46.686 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:46.683+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:46.687 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:46.683+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:46.687 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:46.683+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:46.687 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:46.683+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:46.833 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:46.827+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:46.834 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:46.827+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:46.834 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:46.827+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:46.834 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:46.827+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:47.534 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:47.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:47.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:47.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:47.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:47.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:47.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:47.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:47.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:47.531+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:47.577 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:47.575+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:47.578 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:47.575+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:47.578 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:47.575+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:47.578 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:47.575+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:47.697 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:47.695+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:47.697 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:47.695+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:47.697 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:47.695+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:47.697 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:47.695+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:47.836 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:47.831+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:47.836 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:47.831+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:47.837 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:47.831+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:47.837 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:47.831+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:48.533 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:48.531+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:48.533 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:48.531+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:48.533 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:48.531+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:48.533 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:48.531+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:48.534 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:48.531+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:48.562 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:48.559+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:48.563 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:48.559+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:48.563 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:48.559+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:48.563 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:48.559+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:48.597 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:13:48.595+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7006 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:13:48.746 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:48.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:48.746 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:48.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:48.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:48.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:48.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:48.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:48.794 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:48.791+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:48.795 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:48.791+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:48.795 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:48.791+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:48.795 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:48.791+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:49.530 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:49.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:49.531 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:49.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:49.531 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:49.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:49.531 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:49.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:49.531 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:49.527+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:49.567 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:49.563+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:49.567 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:49.563+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:49.567 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:49.563+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:49.568 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:49.563+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:49.702 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:49.699+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:49.702 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:49.699+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:49.702 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:49.699+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:49.703 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:49.699+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:49.765 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:49.759+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:49.765 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:49.759+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:49.766 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:49.759+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:49.766 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:49.759+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:50.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~217s 2020-05-18T22:13:50.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~217s 2020-05-18T22:13:50.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~217s 2020-05-18T22:13:50.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~217s 2020-05-18T22:13:50.184 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~217s 2020-05-18T22:13:50.184 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~217s 2020-05-18T22:13:50.184 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~217s 2020-05-18T22:13:50.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~217s 2020-05-18T22:13:50.527 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:50.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:50.528 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:50.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:50.528 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:50.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:50.528 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:50.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:50.528 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:50.527+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:50.590 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:50.583+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:50.591 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:50.583+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:50.591 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:50.583+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:50.591 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:50.583+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:50.737 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:50.735+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:50.737 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:50.735+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:50.738 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:50.735+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:50.738 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:50.735+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:50.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:50.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:50.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:50.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:50.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:50.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:50.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:50.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:51.502 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:51.495+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:51.502 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:51.495+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:51.502 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:51.495+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:51.502 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:51.495+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:51.503 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:51.495+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:51.570 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:51.567+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:51.571 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:51.567+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:51.571 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:51.567+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:51.571 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:51.567+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:51.736 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:51.731+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:51.736 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:51.731+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:51.737 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:51.731+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:51.737 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:51.731+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:51.793 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:51.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:51.793 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:51.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:51.793 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:51.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:51.793 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:51.787+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:52.467 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:52.463+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:52.468 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:52.463+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:52.468 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:52.463+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:52.468 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:52.463+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:52.468 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:52.463+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:52.523 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:52.523+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:52.524 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:52.523+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:52.524 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:52.523+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:52.524 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:52.523+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:52.724 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:52.723+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:52.725 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:52.723+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:52.725 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:52.723+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:52.725 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:52.723+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:52.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:52.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:52.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:52.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:52.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:52.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:52.747 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:52.743+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:53.450 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:53.443+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:53.450 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:53.443+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:53.450 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:53.443+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:53.451 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:53.443+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:53.451 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:53.443+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:53.490 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:53.487+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:53.491 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:53.487+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:53.491 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:53.487+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:53.491 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:53.487+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:53.599 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:13:53.595+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7042 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:13:53.712 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:53.711+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:53.713 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:53.711+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:53.713 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:53.711+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:53.714 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:53.711+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:53.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:53.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:53.759 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:53.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:53.760 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:53.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:53.760 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:53.755+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:54.475 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:54.471+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:54.475 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:54.471+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:54.475 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:54.471+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:54.476 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:54.471+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:54.476 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:54.471+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:54.514 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:54.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:54.514 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:54.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:54.515 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:54.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:54.515 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:54.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:54.674 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:54.671+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:54.674 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:54.671+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:54.675 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:54.671+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:54.675 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:54.671+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:54.745 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:54.739+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:54.745 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:54.739+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:54.745 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:54.739+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:54.746 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:54.739+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:55.473 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:55.471+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:55.473 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:55.471+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:55.473 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:55.471+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:55.473 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:55.471+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:55.474 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:55.471+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:55.478 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:55.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:55.478 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:55.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:55.478 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:55.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:55.478 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:55.475+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:55.690 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:55.687+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:55.690 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:55.687+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:55.691 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:55.687+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:55.691 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:55.687+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:55.722 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:55.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:55.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:55.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:55.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:55.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:55.723 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:55.719+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:55.890 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~223s 2020-05-18T22:13:55.891 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~223s 2020-05-18T22:13:55.891 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~223s 2020-05-18T22:13:55.891 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~223s 2020-05-18T22:13:55.891 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~223s 2020-05-18T22:13:55.892 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~223s 2020-05-18T22:13:55.892 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~223s 2020-05-18T22:13:55.892 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~223s 2020-05-18T22:13:56.433 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:56.431+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:56.433 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:56.431+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:56.433 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:56.431+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:56.434 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:56.431+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:56.522 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:56.519+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:56.523 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:56.519+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:56.523 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:56.519+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:56.523 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:56.519+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:56.523 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:56.519+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:56.705 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:56.703+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:56.705 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:56.703+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:56.705 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:56.703+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:56.705 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:56.703+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:56.718 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:56.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:56.718 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:56.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:56.718 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:56.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:56.718 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:56.715+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:57.483 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:57.483+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:57.555 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:57.483+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:57.556 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:57.483+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:57.556 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:57.483+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:57.556 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:57.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:57.557 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:57.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:57.557 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:57.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:57.557 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:57.527+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:57.557 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:57.527+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:57.701 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:57.699+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:57.702 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:57.699+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:57.702 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:57.699+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:57.702 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:57.699+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:57.727 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:57.723+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:57.728 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:57.723+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:57.728 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:57.723+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:57.728 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:57.723+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:57.843 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:13:57.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:13:58.465 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:58.463+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:58.465 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:58.463+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:58.466 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:58.463+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:58.466 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:58.463+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:58.545 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:58.543+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:58.545 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:58.543+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:58.546 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:58.543+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:58.546 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:58.543+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:58.546 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:58.543+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:58.601 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:13:58.599+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7078 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:13:58.730 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:58.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:58.730 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:58.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:58.731 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:58.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:58.731 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:58.727+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:58.740 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:58.735+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:58.741 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:58.735+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:58.741 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:58.735+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:58.741 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:58.735+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:59.458 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:59.455+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:59.458 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:59.455+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:59.459 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:59.455+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:59.459 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:13:59.455+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:13:59.578 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:59.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:59.579 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:59.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:59.579 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:59.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:59.579 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:59.575+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:13:59.579 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:13:59.575+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:13:59.712 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:59.711+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:59.713 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:59.711+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:59.713 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:59.711+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:59.713 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:13:59.711+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:13:59.714 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:59.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:59.714 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:59.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:59.714 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:59.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:13:59.714 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:13:59.711+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:00.463 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:00.463+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:00.463 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:00.463+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:00.463 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:00.463+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:00.464 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:00.463+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:00.551 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:00.547+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:00.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:00.547+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:00.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:00.547+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:00.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:00.547+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:00.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:00.547+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:00.679 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:00.679+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:00.679 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:00.679+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:00.679 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:00.679+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:00.680 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:00.679+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:00.699 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:00.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:00.700 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:00.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:00.700 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:00.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:00.700 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:00.695+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:00.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:14:00.926 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:14:01.513 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:01.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:01.513 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:01.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:01.513 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:01.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:01.514 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:01.511+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:01.557 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:01.555+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:01.558 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:01.555+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:01.558 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:01.555+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:01.558 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:01.555+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:01.558 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:01.555+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:01.598 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~228s 2020-05-18T22:14:01.598 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~228s 2020-05-18T22:14:01.598 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~228s 2020-05-18T22:14:01.599 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~228s 2020-05-18T22:14:01.599 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~228s 2020-05-18T22:14:01.599 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~228s 2020-05-18T22:14:01.599 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~228s 2020-05-18T22:14:01.599 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~228s 2020-05-18T22:14:01.636 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:01.635+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:01.636 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:01.635+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:01.636 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:01.635+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:01.637 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:01.635+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:01.735 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:01.735+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:01.736 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:01.735+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:01.736 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:01.735+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:01.736 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:01.735+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:02.477 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:02.474+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:02.477 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:02.474+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:02.478 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:02.474+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:02.478 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:02.474+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:02.560 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:02.554+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:02.560 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:02.554+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:02.560 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:02.554+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:02.560 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:02.554+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:02.561 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:02.554+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:02.601 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:02.598+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:02.601 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:02.598+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:02.602 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:02.598+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:02.602 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:02.598+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:02.746 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:02.742+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:02.747 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:02.742+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:02.747 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:02.742+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:02.747 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:02.742+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:03.471 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:03.466+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:03.472 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:03.466+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:03.472 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:03.466+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:03.472 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:03.466+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:03.560 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:03.554+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:03.560 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:03.554+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:03.560 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:03.554+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:03.561 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:03.554+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:03.603 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:03.598+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:03.603 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:03.598+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:03.603 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:03.598+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:03.604 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:03.598+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:03.604 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:03.598+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:03.604 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:14:03.598+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7113 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:14:03.772 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:03.766+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:03.773 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:03.766+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:03.773 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:03.766+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:03.773 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:03.766+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:04.429 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:04.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:04.429 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:04.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:04.429 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:04.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:04.429 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:04.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:04.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:04.550+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:04.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:04.550+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:04.553 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:04.550+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:04.553 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:04.550+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:04.558 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:04.554+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:04.558 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:04.554+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:04.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:04.554+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:04.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:04.554+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:04.559 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:04.554+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:04.777 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:04.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:04.777 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:04.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:04.777 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:04.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:04.777 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:04.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:05.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:05.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:05.406 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:05.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:05.406 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:05.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:05.406 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:05.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:05.577 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:05.574+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:05.577 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:05.574+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:05.577 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:05.574+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:05.578 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:05.574+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:05.583 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:05.578+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:05.583 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:05.578+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:05.583 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:05.578+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:05.583 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:05.578+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:05.584 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:05.578+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:05.753 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:05.746+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:05.753 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:05.746+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:05.754 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:05.746+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:05.754 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:05.746+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:06.442 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:06.442+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:06.443 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:06.442+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:06.443 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:06.442+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:06.443 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:06.442+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:06.551 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:06.550+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:06.551 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:06.550+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:06.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:06.550+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:06.552 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:06.550+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:06.556 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:06.550+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:06.556 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:06.550+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:06.556 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:06.550+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:06.556 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:06.550+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:06.557 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:06.550+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:06.735 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:06.734+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:06.736 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:06.734+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:06.736 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:06.734+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:06.736 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:06.734+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:07.305 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~234s 2020-05-18T22:14:07.306 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~234s 2020-05-18T22:14:07.306 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~234s 2020-05-18T22:14:07.306 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~234s 2020-05-18T22:14:07.306 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~234s 2020-05-18T22:14:07.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~234s 2020-05-18T22:14:07.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~234s 2020-05-18T22:14:07.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~234s 2020-05-18T22:14:07.435 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:07.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:07.435 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:07.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:07.435 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:07.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:07.436 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:07.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:07.569 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:07.566+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:07.569 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:07.566+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:07.569 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:07.566+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:07.569 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:07.566+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:07.570 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:07.566+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:07.573 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:07.566+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:07.574 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:07.566+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:07.574 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:07.566+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:07.574 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:07.566+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:07.694 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:07.690+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:07.695 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:07.690+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:07.695 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:07.690+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:07.695 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:07.690+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:08.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:08.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:08.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:08.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:08.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:08.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:08.402 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:08.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:08.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:08.590+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:08.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:08.590+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:08.594 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:08.590+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:08.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:08.590+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:08.595 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:08.590+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:08.605 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:14:08.598+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7148 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:14:08.606 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:08.598+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:08.606 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:08.598+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:08.606 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:08.598+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:08.607 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:08.598+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:08.731 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:08.726+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:08.732 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:08.726+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:08.732 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:08.726+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:08.732 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:08.726+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:09.441 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:09.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:09.442 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:09.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:09.442 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:09.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:09.442 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:09.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:09.561 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:09.554+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:09.561 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:09.554+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:09.561 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:09.554+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:09.561 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:09.554+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:09.584 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:09.582+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:09.585 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:09.582+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:09.585 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:09.582+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:09.585 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:09.582+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:09.585 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:09.582+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:09.691 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:09.690+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:09.692 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:09.690+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:09.692 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:09.690+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:09.692 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:09.690+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:10.425 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:10.422+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:10.425 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:10.422+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:10.425 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:10.422+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:10.426 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:10.422+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:10.571 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:10.570+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:10.572 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:10.570+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:10.572 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:10.570+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:10.572 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:10.570+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:10.572 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:10.570+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:10.574 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:10.570+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:10.574 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:10.570+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:10.575 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:10.570+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:10.575 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:10.570+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:10.719 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:10.718+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:10.719 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:10.718+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:10.720 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:10.718+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:10.720 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:10.718+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:11.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:11.378+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:11.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:11.378+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:11.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:11.378+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:11.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:11.378+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:11.581 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:11.574+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:11.581 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:11.574+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:11.582 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:11.574+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:11.582 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:11.574+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:11.582 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:11.574+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:11.624 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:11.618+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:11.624 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:11.618+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:11.625 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:11.618+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:11.625 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:11.618+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:11.724 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:11.722+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:11.724 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:11.722+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:11.725 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:11.722+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:11.725 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:11.722+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:12.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:12.370+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:12.373 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:12.370+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:12.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:12.370+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:12.374 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:12.370+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:12.562 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:12.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:12.562 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:12.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:12.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:12.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:12.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:12.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:12.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:12.558+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:12.573 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:12.566+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:12.573 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:12.566+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:12.573 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:12.566+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:12.574 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:12.566+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:12.739 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:12.738+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:12.739 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:12.738+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:12.740 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:12.738+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:12.740 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:12.738+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:13.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~240s 2020-05-18T22:14:13.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~240s 2020-05-18T22:14:13.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~240s 2020-05-18T22:14:13.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~240s 2020-05-18T22:14:13.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~240s 2020-05-18T22:14:13.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~240s 2020-05-18T22:14:13.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~240s 2020-05-18T22:14:13.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~240s 2020-05-18T22:14:13.378 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:13.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:13.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:13.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:13.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:13.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:13.379 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:13.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:13.528 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:13.526+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:13.529 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:13.526+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:13.529 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:13.526+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:13.529 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:13.526+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:13.570 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:13.566+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:13.570 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:13.566+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:13.571 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:13.566+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:13.571 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:13.566+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:13.571 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:13.566+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:13.607 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:14:13.602+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7184 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:14:13.777 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:13.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:13.778 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:13.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:13.778 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:13.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:13.778 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:13.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:14.362 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:14.358+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:14.363 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:14.358+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:14.363 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:14.358+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:14.363 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:14.358+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:14.511 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:14.506+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:14.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:14.506+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:14.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:14.506+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:14.512 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:14.506+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:14.547 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:14.546+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:14.547 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:14.546+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:14.548 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:14.546+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:14.548 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:14.546+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:14.548 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:14.546+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:14.814 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:14.810+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:14.814 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:14.810+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:14.815 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:14.810+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:14.815 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:14.810+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:15.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:15.378+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:15.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:15.378+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:15.382 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:15.378+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:15.383 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:15.378+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:15.538 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:15.534+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:15.538 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:15.534+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:15.539 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:15.534+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:15.539 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:15.534+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:15.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:15.582+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:15.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:15.582+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:15.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:15.582+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:15.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:15.582+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:15.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:15.582+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:15.845 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:15.838+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:15.845 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:15.838+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:15.845 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:15.838+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:15.846 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:15.838+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:16.394 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:16.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:16.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:16.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:16.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:16.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:16.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:16.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:16.492 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:16.490+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:16.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:16.490+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:16.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:16.490+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:16.493 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:16.490+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:16.591 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:16.590+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:16.591 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:16.590+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:16.591 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:16.590+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:16.591 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:16.590+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:16.592 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:16.590+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:16.807 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:16.802+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:16.808 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:16.802+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:16.808 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:16.802+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:16.808 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:16.802+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:17.421 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:17.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:17.421 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:17.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:17.421 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:17.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:17.422 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:17.414+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:17.494 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:17.494+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:17.495 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:17.494+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:17.495 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:17.494+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:17.495 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:17.494+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:17.551 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:17.546+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:17.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:17.546+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:17.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:17.546+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:17.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:17.546+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:17.552 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:17.546+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:17.814 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:17.810+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:17.815 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:17.810+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:17.815 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:17.810+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:17.815 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:17.810+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:18.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:18.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:18.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:18.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:18.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:18.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:18.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:18.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:18.498 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:18.498+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:18.499 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:18.498+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:18.499 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:18.498+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:18.499 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:18.498+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:18.571 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:18.566+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:18.571 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:18.566+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:18.571 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:18.566+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:18.572 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:18.566+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:18.572 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:18.566+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:18.608 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:14:18.602+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7219 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:14:18.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~245s 2020-05-18T22:14:18.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~245s 2020-05-18T22:14:18.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~245s 2020-05-18T22:14:18.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~245s 2020-05-18T22:14:18.721 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~245s 2020-05-18T22:14:18.721 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~245s 2020-05-18T22:14:18.721 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~245s 2020-05-18T22:14:18.721 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~245s 2020-05-18T22:14:18.862 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:18.858+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:18.862 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:18.858+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:18.863 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:18.858+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:18.863 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:18.858+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:19.349 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:19.342+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:19.349 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:19.342+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:19.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:19.342+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:19.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:19.342+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:19.463 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:19.462+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:19.464 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:19.462+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:19.464 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:19.462+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:19.464 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:19.462+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:19.565 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:19.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:19.566 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:19.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:19.566 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:19.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:19.566 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:19.558+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:19.566 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:19.558+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:19.827 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:19.822+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:19.827 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:19.822+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:19.827 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:19.822+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:19.828 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:19.822+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:20.375 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:20.370+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:20.376 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:20.370+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:20.376 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:20.370+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:20.376 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:20.370+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:20.434 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:20.430+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:20.434 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:20.430+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:20.434 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:20.430+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:20.434 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:20.430+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:20.545 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:20.542+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:20.545 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:20.542+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:20.545 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:20.542+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:20.546 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:20.542+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:20.546 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:20.542+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:20.794 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:20.790+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:20.795 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:20.790+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:20.795 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:20.790+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:20.795 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:20.790+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:21.393 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:21.390+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:21.393 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:21.390+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:21.394 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:21.390+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:21.394 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:21.390+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:21.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:21.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:21.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:21.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:21.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:21.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:21.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:21.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:21.539 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:21.538+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:21.539 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:21.538+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:21.539 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:21.538+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:21.539 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:21.538+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:21.540 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:21.538+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:21.770 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:21.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:21.771 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:21.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:21.771 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:21.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:21.771 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:21.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:22.381 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:22.374+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:22.381 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:22.374+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:22.382 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:22.374+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:22.382 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:22.374+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:22.411 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:22.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:22.411 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:22.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:22.412 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:22.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:22.412 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:22.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:22.512 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:22.510+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:22.513 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:22.510+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:22.513 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:22.510+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:22.513 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:22.510+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:22.513 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:22.510+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:22.750 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:22.746+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:22.751 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:22.746+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:22.751 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:22.746+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:22.751 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:22.746+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:23.380 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:23.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:23.380 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:23.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:23.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:23.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:23.381 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:23.374+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:23.419 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:23.414+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:23.420 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:23.414+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:23.420 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:23.414+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:23.420 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:23.414+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:23.494 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:23.490+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:23.495 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:23.490+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:23.495 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:23.490+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:23.495 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:23.490+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:23.495 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:23.490+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:23.610 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:14:23.606+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7255 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:14:23.800 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:23.794+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:23.800 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:23.794+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:23.800 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:23.794+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:23.801 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:23.794+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:24.412 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:24.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:24.412 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:24.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:24.412 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:24.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:24.412 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:24.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:24.428 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~251s 2020-05-18T22:14:24.428 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~251s 2020-05-18T22:14:24.429 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~251s 2020-05-18T22:14:24.429 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~251s 2020-05-18T22:14:24.429 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~251s 2020-05-18T22:14:24.429 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~251s 2020-05-18T22:14:24.429 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~251s 2020-05-18T22:14:24.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~251s 2020-05-18T22:14:24.451 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:24.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:24.451 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:24.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:24.451 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:24.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:24.452 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:24.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:24.527 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:24.526+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:24.528 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:24.526+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:24.528 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:24.526+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:24.528 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:24.526+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:24.529 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:24.526+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:24.769 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:24.762+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:24.769 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:24.762+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:24.770 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:24.762+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:24.770 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:24.762+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:25.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:25.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:25.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:25.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:25.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:25.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:25.387 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:25.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:25.473 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:25.470+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:25.473 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:25.470+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:25.474 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:25.470+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:25.474 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:25.470+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:25.539 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:25.538+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:25.539 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:25.538+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:25.540 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:25.538+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:25.540 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:25.538+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:25.540 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:25.538+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:25.732 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:25.730+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:25.732 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:25.730+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:25.732 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:25.730+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:25.733 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:25.730+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:26.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:26.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:26.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:26.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:26.405 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:26.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:26.406 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:26.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:26.455 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:26.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:26.456 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:26.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:26.456 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:26.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:26.456 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:26.450+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:26.542 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:26.542+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:26.543 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:26.542+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:26.543 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:26.542+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:26.543 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:26.542+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:26.543 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:26.542+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:26.724 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:26.722+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:26.724 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:26.722+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:26.724 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:26.722+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:26.725 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:26.722+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:27.385 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:27.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:27.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:27.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:27.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:27.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:27.386 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:27.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:27.459 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:27.458+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:27.459 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:27.458+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:27.459 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:27.458+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:27.459 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:27.458+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:27.562 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:27.562+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:27.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:27.562+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:27.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:27.562+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:27.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:27.562+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:27.563 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:27.562+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:27.736 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:27.730+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:27.736 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:27.730+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:27.736 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:27.730+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:27.737 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:27.730+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:28.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:28.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:28.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:28.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:28.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:28.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:28.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:28.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:28.466 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:28.466+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:28.467 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:28.466+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:28.467 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:28.466+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:28.467 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:28.466+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:28.532 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:28.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:28.532 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:28.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:28.532 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:28.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:28.532 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:28.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:28.533 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:28.530+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:28.612 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:14:28.610+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7289 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:14:28.748 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:28.746+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:28.749 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:28.746+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:28.749 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:28.746+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:28.749 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:28.746+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:29.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:29.418+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:29.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:29.418+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:29.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:29.418+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:29.423 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:29.418+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:29.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:29.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:29.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:29.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:29.428 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:29.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:29.428 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:29.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:29.486 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:29.482+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:29.486 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:29.482+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:29.487 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:29.482+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:29.487 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:29.482+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:29.487 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:29.482+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:29.769 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:29.766+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:29.769 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:29.766+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:29.769 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:29.766+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:29.770 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:29.766+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:30.135 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~257s 2020-05-18T22:14:30.135 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~257s 2020-05-18T22:14:30.135 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~257s 2020-05-18T22:14:30.135 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~257s 2020-05-18T22:14:30.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~257s 2020-05-18T22:14:30.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~257s 2020-05-18T22:14:30.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~257s 2020-05-18T22:14:30.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~257s 2020-05-18T22:14:30.380 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:30.378+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:30.380 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:30.378+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:30.380 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:30.378+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:30.381 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:30.378+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:30.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:30.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:30.394 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:30.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:30.394 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:30.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:30.394 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:30.390+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:30.466 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:30.466+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:30.466 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:30.466+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:30.467 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:30.466+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:30.467 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:30.466+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:30.467 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:30.466+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:30.811 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:30.810+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:30.812 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:30.810+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:30.812 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:30.810+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:30.812 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:30.810+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:30.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:14:30.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:14:31.331 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:31.330+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:31.331 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:31.330+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:31.332 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:31.330+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:31.332 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:31.330+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:31.357 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:31.354+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:31.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:31.354+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:31.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:31.354+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:31.358 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:31.354+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:31.468 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:31.466+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:31.468 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:31.466+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:31.469 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:31.466+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:31.469 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:31.466+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:31.469 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:31.466+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:31.770 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:31.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:31.771 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:31.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:31.771 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:31.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:31.771 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:31.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:32.286 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:32.286+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:32.287 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:32.286+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:32.287 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:32.286+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:32.287 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:32.286+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:32.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:32.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:32.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:32.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:32.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:32.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:32.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:32.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:32.513 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:32.510+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:32.513 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:32.510+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:32.513 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:32.510+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:32.513 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:32.510+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:32.514 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:32.510+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:32.815 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:32.814+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:32.815 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:32.814+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:32.815 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:32.814+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:32.816 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:32.814+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:33.262 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:33.258+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:33.262 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:33.258+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:33.262 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:33.258+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:33.263 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:33.258+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:33.384 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:33.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:33.385 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:33.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:33.385 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:33.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:33.385 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:33.382+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:33.515 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:33.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:33.516 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:33.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:33.516 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:33.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:33.516 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:33.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:33.516 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:33.514+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:33.614 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:14:33.610+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7326 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:14:33.771 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:33.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:33.772 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:33.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:33.772 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:33.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:33.772 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:33.770+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:34.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:14:34.046 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:14:34.298 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:34.294+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:34.298 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:34.294+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:34.298 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:34.294+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:34.299 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:34.294+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:34.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:34.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:34.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:34.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:34.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:34.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:34.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:34.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:34.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:34.534+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:34.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:34.534+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:34.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:34.534+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:34.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:34.534+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:34.536 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:34.534+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:34.755 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:34.754+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:34.756 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:34.754+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:34.756 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:34.754+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:34.756 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:34.754+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:35.250 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:35.250+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:35.250 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:35.250+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:35.251 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:35.250+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:35.251 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:35.250+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:35.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:35.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:35.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:35.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:35.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:35.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:35.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:35.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:35.514 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:35.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:35.515 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:35.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:35.515 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:35.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:35.516 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:35.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:35.516 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:35.514+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:35.708 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:35.706+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:35.709 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:35.706+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:35.709 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:35.706+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:35.710 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:35.706+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:35.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~263s 2020-05-18T22:14:35.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~263s 2020-05-18T22:14:35.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~263s 2020-05-18T22:14:35.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~263s 2020-05-18T22:14:35.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~263s 2020-05-18T22:14:35.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~263s 2020-05-18T22:14:35.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~263s 2020-05-18T22:14:35.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~263s 2020-05-18T22:14:36.259 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:36.258+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:36.260 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:36.258+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:36.260 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:36.258+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:36.260 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:36.258+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:36.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:36.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:36.395 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:36.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:36.396 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:36.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:36.396 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:36.394+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:36.487 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:36.486+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:36.488 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:36.486+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:36.488 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:36.486+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:36.488 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:36.486+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:36.488 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:36.486+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:36.673 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:36.670+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:36.673 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:36.670+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:36.674 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:36.670+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:36.674 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:36.670+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:37.242 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:37.242+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:37.242 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:37.242+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:37.243 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:37.242+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:37.243 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:37.242+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:37.403 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:37.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:37.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:37.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:37.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:37.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:37.404 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:37.402+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:37.501 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:37.498+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:37.501 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:37.498+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:37.502 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:37.498+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:37.502 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:37.498+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:37.502 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:37.498+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:37.682 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:37.682+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:37.682 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:37.682+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:37.682 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:37.682+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:37.683 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:37.682+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:38.242 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:38.242+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:38.243 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:38.242+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:38.243 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:38.242+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:38.243 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:38.242+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:38.412 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:38.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:38.413 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:38.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:38.413 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:38.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:38.413 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:38.410+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:38.547 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:38.546+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:38.547 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:38.546+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:38.547 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:38.546+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:38.548 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:38.546+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:38.548 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:38.546+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:38.616 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:14:38.614+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7361 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:14:38.725 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:38.722+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:38.726 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:38.722+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:38.726 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:38.722+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:38.726 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:38.722+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:39.242 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:39.238+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:39.242 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:39.238+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:39.243 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:39.238+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:39.243 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:39.238+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:39.423 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:39.422+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:39.424 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:39.422+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:39.424 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:39.422+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:39.424 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:39.422+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:39.540 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:39.538+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:39.541 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:39.538+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:39.541 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:39.538+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:39.542 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:39.538+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:39.542 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:39.538+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:39.695 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:39.694+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:39.696 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:39.694+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:39.696 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:39.694+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:39.696 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:39.694+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:40.195 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:40.194+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:40.195 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:40.194+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:40.195 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:40.194+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:40.196 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:40.194+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:40.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:40.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:40.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:40.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:40.427 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:40.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:40.428 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:40.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:40.504 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:40.502+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:40.504 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:40.502+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:40.504 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:40.502+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:40.505 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:40.502+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:40.505 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:40.502+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:40.703 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:40.702+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:40.704 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:40.702+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:40.704 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:40.702+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:40.704 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:40.702+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:41.219 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:41.218+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:41.220 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:41.218+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:41.220 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:41.218+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:41.220 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:41.218+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:41.437 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:41.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:41.438 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:41.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:41.438 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:41.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:41.438 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:41.434+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:41.481 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:41.478+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:41.482 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:41.478+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:41.482 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:41.478+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:41.482 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:41.478+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:41.482 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:41.478+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:41.558 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~268s 2020-05-18T22:14:41.558 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~268s 2020-05-18T22:14:41.559 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~268s 2020-05-18T22:14:41.559 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~268s 2020-05-18T22:14:41.559 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~268s 2020-05-18T22:14:41.559 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~268s 2020-05-18T22:14:41.560 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~268s 2020-05-18T22:14:41.560 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~268s 2020-05-18T22:14:41.701 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:41.698+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:41.702 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:41.698+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:41.702 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:41.698+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:41.702 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:41.698+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:42.194 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:42.194+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:42.195 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:42.194+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:42.195 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:42.194+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:42.195 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:42.194+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:42.458 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:42.454+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:42.458 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:42.454+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:42.459 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:42.454+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:42.459 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:42.454+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:42.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:42.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:42.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:42.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:42.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:42.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:42.536 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:42.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:42.536 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:42.530+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:42.702 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:42.702+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:42.703 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:42.702+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:42.703 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:42.702+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:42.703 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:42.702+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:43.230 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:43.230+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:43.231 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:43.230+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:43.231 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:43.230+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:43.231 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:43.230+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:43.429 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:43.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:43.430 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:43.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:43.430 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:43.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:43.430 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:43.426+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:43.486 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:43.486+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:43.487 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:43.486+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:43.487 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:43.486+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:43.487 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:43.486+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:43.488 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:43.486+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:43.619 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:14:43.618+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7398 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:14:43.696 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:43.694+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:43.696 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:43.694+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:43.696 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:43.694+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:43.696 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:43.694+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:44.200 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:44.198+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:44.201 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:44.198+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:44.201 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:44.198+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:44.201 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:44.198+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:44.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:44.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:44.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:44.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:44.399 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:44.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:44.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:44.398+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:44.474 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:44.474+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:44.475 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:44.474+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:44.475 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:44.474+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:44.475 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:44.474+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:44.476 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:44.474+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:44.655 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:44.654+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:44.655 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:44.654+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:44.655 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:44.654+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:44.656 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:44.654+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:45.166 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:45.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:45.167 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:45.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:45.167 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:45.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:45.167 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:45.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:45.371 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:45.370+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:45.371 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:45.370+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:45.371 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:45.370+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:45.372 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:45.370+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:45.515 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:45.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:45.515 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:45.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:45.515 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:45.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:45.516 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:45.514+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:45.516 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:45.514+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:45.652 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:45.650+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:45.653 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:45.650+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:45.653 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:45.650+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:45.653 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:45.650+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:46.162 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:46.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:46.163 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:46.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:46.163 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:46.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:46.163 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:46.158+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:46.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:46.358+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:46.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:46.358+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:46.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:46.358+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:46.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:46.358+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:46.498 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:46.498+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:46.498 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:46.498+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:46.499 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:46.498+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:46.499 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:46.498+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:46.499 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:46.498+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:46.681 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:46.678+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:46.681 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:46.678+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:46.681 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:46.678+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:46.682 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:46.678+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:47.148 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:47.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:47.148 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:47.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:47.149 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:47.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:47.149 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:47.146+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:47.266 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~274s 2020-05-18T22:14:47.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~274s 2020-05-18T22:14:47.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~274s 2020-05-18T22:14:47.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~274s 2020-05-18T22:14:47.268 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~274s 2020-05-18T22:14:47.268 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~274s 2020-05-18T22:14:47.268 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~274s 2020-05-18T22:14:47.268 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~274s 2020-05-18T22:14:47.323 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:47.322+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:47.324 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:47.322+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:47.324 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:47.322+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:47.324 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:47.322+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:47.534 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:47.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:47.534 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:47.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:47.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:47.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:47.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:47.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:47.535 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:47.534+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:47.668 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:47.666+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:47.668 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:47.666+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:47.668 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:47.666+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:47.669 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:47.666+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:48.185 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:48.182+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:48.185 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:48.182+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:48.185 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:48.182+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:48.186 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:48.182+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:48.340 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:48.338+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:48.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:48.338+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:48.341 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:48.338+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:48.342 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:48.338+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:48.532 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:48.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:48.532 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:48.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:48.533 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:48.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:48.533 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:48.530+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:48.534 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:48.530+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:48.620 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:14:48.618+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7432 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:14:48.659 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:48.658+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:48.660 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:48.658+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:48.660 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:48.658+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:48.660 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:48.658+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:49.141 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:49.138+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:49.142 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:49.138+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:49.142 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:49.138+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:49.142 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:49.138+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:49.324 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:49.322+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:49.325 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:49.322+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:49.325 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:49.322+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:49.325 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:49.322+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:49.581 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:49.578+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:49.581 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:49.578+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:49.582 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:49.578+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:49.582 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:49.578+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:49.582 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:49.578+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:49.660 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:49.658+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:49.660 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:49.658+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:49.660 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:49.658+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:49.660 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:49.658+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:50.167 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:50.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:50.168 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:50.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:50.168 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:50.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:50.168 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:50.166+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:50.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:50.314+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:50.316 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:50.314+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:50.316 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:50.314+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:50.316 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:50.314+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:50.606 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:50.606+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:50.607 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:50.606+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:50.607 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:50.606+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:50.607 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:50.606+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:50.607 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:50.606+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:50.697 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:50.694+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:50.697 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:50.694+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:50.697 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:50.694+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:50.697 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:50.694+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:51.151 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:51.150+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:51.152 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:51.150+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:51.152 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:51.150+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:51.152 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:51.150+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:51.345 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:51.341+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:51.345 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:51.341+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:51.346 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:51.341+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:51.346 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:51.341+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:51.620 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:51.617+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:51.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:51.617+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:51.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:51.617+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:51.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:51.617+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:51.621 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:51.617+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:51.697 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:51.693+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:51.698 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:51.693+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:51.698 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:51.693+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:51.698 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:51.693+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:52.124 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:52.121+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:52.124 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:52.121+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:52.125 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:52.121+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:52.125 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:52.121+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:52.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:52.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:52.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:52.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:52.315 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:52.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:52.316 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:52.313+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:52.602 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:52.597+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:52.602 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:52.597+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:52.602 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:52.597+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:52.602 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:52.597+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:52.603 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:52.597+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:52.754 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:52.745+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:52.754 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:52.745+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:52.754 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:52.745+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:52.755 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:52.745+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:52.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~280s 2020-05-18T22:14:52.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~280s 2020-05-18T22:14:52.975 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~280s 2020-05-18T22:14:52.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~280s 2020-05-18T22:14:52.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~280s 2020-05-18T22:14:52.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~280s 2020-05-18T22:14:52.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~280s 2020-05-18T22:14:52.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~280s 2020-05-18T22:14:53.086 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:53.085+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:53.087 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:53.085+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:53.087 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:53.085+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:53.087 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:53.085+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:53.322 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:53.321+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:53.322 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:53.321+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:53.322 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:53.321+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:53.323 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:53.321+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:53.573 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:53.569+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:53.573 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:53.569+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:53.574 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:53.569+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:53.574 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:53.569+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:53.574 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:53.569+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:53.623 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:14:53.621+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7468 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:14:53.742 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:53.741+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:53.742 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:53.741+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:53.742 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:53.741+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:53.743 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:53.741+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:54.087 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:54.085+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:54.088 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:54.085+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:54.088 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:54.085+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:54.088 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:54.085+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:54.359 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:54.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:54.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:54.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:54.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:54.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:54.360 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:54.357+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:54.526 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:54.521+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:54.526 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:54.521+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:54.526 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:54.521+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:54.527 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:54.521+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:54.527 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:54.525+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:54.785 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:54.781+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:54.785 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:54.781+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:54.786 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:54.781+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:54.786 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:54.781+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:55.119 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:55.117+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:55.120 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:55.117+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:55.120 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:55.117+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:55.120 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:55.117+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:55.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:55.389+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:55.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:55.389+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:55.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:55.389+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:55.393 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:55.389+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:55.550 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:55.549+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:55.550 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:55.549+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:55.550 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:55.549+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:55.551 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:55.549+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:55.551 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:55.549+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:55.766 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:55.761+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:55.766 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:55.761+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:55.767 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:55.761+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:55.767 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:55.761+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:56.105 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:56.101+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:56.106 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:56.101+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:56.106 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:56.101+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:56.106 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:56.101+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:56.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:56.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:56.400 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:56.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:56.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:56.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:56.401 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:56.397+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:56.585 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:56.581+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:56.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:56.581+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:56.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:56.581+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:56.586 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:56.581+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:56.587 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:56.581+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:56.811 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:56.809+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:56.811 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:56.809+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:56.811 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:56.809+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:56.812 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:56.809+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:57.061 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:57.057+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:57.062 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:57.057+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:57.062 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:57.057+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:57.062 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:57.057+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:57.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:57.349+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:57.352 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:57.349+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:57.353 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:57.349+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:57.353 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:57.349+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:57.599 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:57.597+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:57.600 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:57.597+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:57.600 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:57.597+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:57.600 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:57.597+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:14:57.601 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:14:57.597+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:14:57.833 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:57.829+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:57.833 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:57.829+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:57.834 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:57.829+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:57.834 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:14:57.829+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:14:58.087 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:58.085+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:58.087 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:58.085+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:58.088 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:58.085+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:58.088 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:14:58.085+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:14:58.350 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:58.349+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:58.351 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:58.349+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:58.351 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:58.349+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:58.351 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:14:58.349+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:14:58.683 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~285s 2020-05-18T22:14:58.683 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~285s 2020-05-18T22:14:58.683 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~285s 2020-05-18T22:14:58.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~285s 2020-05-18T22:14:58.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~285s 2020-05-18T22:14:58.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~285s 2020-05-18T22:14:58.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~285s 2020-05-18T22:14:58.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~285s 2020-05-18T22:15:04.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:15:04.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:15:04.391 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~291s 2020-05-18T22:15:04.392 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~291s 2020-05-18T22:15:04.392 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~291s 2020-05-18T22:15:04.392 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~291s 2020-05-18T22:15:04.392 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~291s 2020-05-18T22:15:04.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~291s 2020-05-18T22:15:04.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~291s 2020-05-18T22:15:04.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~291s 2020-05-18T22:15:07.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:15:07.103 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:15:10.100 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~297s 2020-05-18T22:15:10.100 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~297s 2020-05-18T22:15:10.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~297s 2020-05-18T22:15:10.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~297s 2020-05-18T22:15:10.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~297s 2020-05-18T22:15:10.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~297s 2020-05-18T22:15:10.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~297s 2020-05-18T22:15:10.102 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~297s 2020-05-18T22:15:15.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~303s 2020-05-18T22:15:15.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~303s 2020-05-18T22:15:15.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~303s 2020-05-18T22:15:15.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~303s 2020-05-18T22:15:15.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~303s 2020-05-18T22:15:15.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~303s 2020-05-18T22:15:15.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~303s 2020-05-18T22:15:15.809 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~303s 2020-05-18T22:15:15.809 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2020-05-18T22:15:15.810 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2020-05-18T22:15:15.810 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-05-18T22:15:15.810 INFO:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-05-18T22:15:18.587 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:15:18.577+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:15:18.587 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:15:18.577+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:15:18.588 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:15:18.577+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:15:18.588 INFO:tasks.ceph.osd.2.smithi196.stderr:2020-05-18T22:15:18.577+0000 7fe8a1db9700 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:18.639394+0000 (oldest deadline 2020-05-18T22:05:38.639394+0000) 2020-05-18T22:15:18.588 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:15:18.581+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:15:18.588 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:15:18.581+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:15:18.589 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:15:18.581+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:15:18.589 INFO:tasks.ceph.osd.3.smithi196.stderr:2020-05-18T22:15:18.581+0000 7f1349e24700 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:19.040261+0000 (oldest deadline 2020-05-18T22:05:39.040261+0000) 2020-05-18T22:15:18.590 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:15:18.577+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:15:18.590 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:15:18.577+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:15:18.590 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:15:18.577+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:15:18.590 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:15:18.577+0000 7f73dfd23700 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.718739+0000 (oldest deadline 2020-05-18T22:05:35.718739+0000) 2020-05-18T22:15:18.591 INFO:tasks.ceph.osd.0.smithi196.stderr:2020-05-18T22:15:18.577+0000 7f73dfd23700 -1 osd.0 22 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4666.0:3216 4.3 4:d60f17ec:::100000003b8.00000016:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e22) 2020-05-18T22:15:18.591 INFO:tasks.ceph.mon.a.smithi196.stderr:2020-05-18T22:15:18.581+0000 7ff885381700 -1 mon.a@1(probing) e1 get_health_metrics reporting 7624 slow ops, oldest is mgrbeacon mgr.x(d048673a-3d91-4cc6-882d-e9cc70e7cd6c,4111, , 0) 2020-05-18T22:15:18.592 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:15:18.581+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6824 osd.4 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:15:18.592 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:15:18.581+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6825 osd.5 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:15:18.592 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:15:18.581+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6809 osd.6 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:15:18.592 INFO:tasks.ceph.osd.1.smithi196.stderr:2020-05-18T22:15:18.581+0000 7fea8ad76700 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.80:6807 osd.7 ever on either front or back, first ping sent 2020-05-18T22:05:15.721326+0000 (oldest deadline 2020-05-18T22:05:35.721326+0000) 2020-05-18T22:15:18.605 INFO:tasks.workunit.client.0.smithi196.stderr:./data/datafile110: Software caused connection abort 2020-05-18T22:15:18.713 INFO:teuthology.orchestra.run.smithi196.stderr:fusermount: failed to unmount /home/ubuntu/cephtest/mnt.0: Device or resource busy 2020-05-18T22:15:18.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-05-18T22:15:18.716 INFO:tasks.cephfs.fuse_mount:Failed to unmount ceph-fuse on ubuntu@smithi196.front.sepia.ceph.com, aborting... 2020-05-18T22:15:18.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-05-18T22:15:18.716 INFO:teuthology.orchestra.run.smithi196:> sudo PATH=/usr/sbin:$PATH lsof ; ps auxf 2020-05-18T22:15:37.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:15:37.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:15:40.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:15:40.159 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:16:10.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:16:10.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:16:13.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:16:13.247 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:16:43.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:16:43.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:16:46.302 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:16:46.303 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:17:16.304 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:17:16.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:17:19.358 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:17:19.359 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:17:49.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:17:49.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:17:52.414 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:17:52.415 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:18:22.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:18:22.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:18:25.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:18:25.470 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:18:55.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:18:55.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:18:58.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:18:58.527 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:19:28.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:19:28.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:19:31.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:19:31.582 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:20:01.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:20:01.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:20:04.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:20:04.639 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:20:34.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:20:34.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:20:37.698 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:20:37.698 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:21:07.700 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:21:07.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:21:10.782 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:21:10.783 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:21:40.784 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:21:40.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:21:43.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:21:43.839 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:22:13.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:22:13.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:22:16.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:22:16.894 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:22:46.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:22:46.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:22:49.954 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:22:49.955 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:23:19.957 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:23:19.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:23:23.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:23:23.039 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:23:53.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:23:53.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:23:56.098 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:23:56.099 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:24:26.100 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:24:26.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:24:29.182 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:24:29.183 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:24:59.184 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:24:59.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:25:02.238 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:25:02.239 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:25:32.240 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:25:32.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:25:35.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:25:35.295 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:26:05.296 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:26:05.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:26:08.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:26:08.351 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:26:38.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:26:38.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:26:41.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:26:41.407 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:27:11.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:27:11.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:27:14.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:27:14.462 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:27:44.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:27:44.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:27:47.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:27:47.518 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:28:17.521 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:28:17.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:28:20.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:28:20.575 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:28:50.579 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:28:50.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:28:53.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:28:53.663 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:29:23.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:29:23.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:29:26.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:29:26.719 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:29:56.721 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:29:56.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:29:59.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:29:59.774 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:30:18.723 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 343, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 275, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2020-05-18T22:30:18.728 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 343, in copy_file_to copy_to_log(src, logger, capture=stream) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 275, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2020-05-18T22:30:29.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:30:29.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:30:32.830 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:30:32.831 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:31:02.833 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:31:02.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:31:05.890 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:31:05.891 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:31:35.893 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:31:35.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:31:38.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:31:38.975 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:32:08.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:32:08.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:32:12.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:32:12.030 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:32:42.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:32:42.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:32:45.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:32:45.087 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:33:15.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:33:15.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:33:18.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:33:18.142 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:33:48.143 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:33:48.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:33:51.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:33:51.231 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:34:21.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:34:21.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:34:24.286 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:34:24.287 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:34:54.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:34:54.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:34:57.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:34:57.343 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:35:27.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:35:27.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:35:30.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:35:30.399 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:36:00.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:36:00.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:36:03.459 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:36:03.460 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:36:33.462 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:36:33.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:36:36.542 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:36:36.543 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:37:06.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:37:06.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:37:09.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:37:09.631 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:37:39.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:37:39.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:37:42.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:37:42.687 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:38:12.689 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:38:12.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:38:15.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:38:15.743 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:38:45.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:38:45.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:38:48.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:38:48.799 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:39:18.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:39:18.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:39:21.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:39:21.854 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:39:51.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:39:51.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:39:54.914 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:39:54.915 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:40:24.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:40:24.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:40:28.002 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:40:28.003 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:40:58.004 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:40:58.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:41:01.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:41:01.086 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:41:31.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:41:31.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:41:34.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:41:34.143 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:42:04.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:42:04.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:42:07.202 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:42:07.203 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:42:37.204 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:42:37.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:42:40.286 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:42:40.287 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:43:10.288 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:43:10.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:43:13.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:43:13.343 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:43:43.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:43:43.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:43:46.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:43:46.398 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:44:16.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:44:16.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:44:19.459 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:44:19.459 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:44:49.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:44:49.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:44:52.542 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:44:52.543 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:45:22.544 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:45:22.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:45:25.598 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:45:25.599 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:45:55.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:45:55.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:45:58.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:45:58.655 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:46:28.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:46:28.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:46:31.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:46:31.743 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:47:01.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:47:01.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:47:04.802 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:47:04.803 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:47:34.804 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:47:34.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:47:37.886 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:47:37.887 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:48:07.889 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:48:07.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:48:10.946 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:48:10.947 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:48:40.949 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:48:40.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:48:44.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:48:44.031 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:49:14.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:49:14.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:49:17.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:49:17.087 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:49:47.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:49:47.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:49:50.146 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:49:50.147 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:50:20.148 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:50:20.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:50:23.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:50:23.230 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:50:53.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:50:53.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:50:56.290 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:50:56.291 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:51:26.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:51:26.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:51:29.374 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:51:29.375 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:51:59.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:51:59.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:52:02.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:52:02.432 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:52:32.434 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:52:32.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:52:35.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:52:35.519 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:53:05.521 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:53:05.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:53:08.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:53:08.575 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:53:38.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:53:38.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:53:41.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:53:41.631 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:54:11.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:54:11.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:54:14.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:54:14.687 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:54:44.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:54:44.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:54:47.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:54:47.743 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:55:17.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:55:17.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:55:20.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:55:20.798 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:55:50.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:55:50.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:55:53.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:55:53.855 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:56:23.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:56:23.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:56:26.914 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:56:26.915 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:56:56.917 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:56:56.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:56:59.998 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:56:59.999 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:57:30.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:57:30.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:57:33.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:57:33.055 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:58:03.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:58:03.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:58:06.110 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:58:06.111 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:58:36.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:58:36.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:58:39.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:58:39.167 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:59:09.169 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:59:09.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:59:12.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:59:12.223 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T22:59:42.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T22:59:42.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T22:59:45.281 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T22:59:45.282 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:00:15.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:00:15.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:00:18.370 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:00:18.371 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:00:48.373 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:00:48.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:00:51.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:00:51.455 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:01:21.463 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:01:21.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:01:24.546 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:01:24.547 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:01:54.548 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:01:54.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:01:57.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:01:57.631 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:02:27.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:02:27.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:02:30.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:02:30.686 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:03:00.689 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:03:00.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:03:03.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:03:03.743 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:03:33.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:03:33.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:03:36.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:03:36.799 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:04:06.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:04:06.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:04:09.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:04:09.855 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:04:39.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:04:39.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:04:42.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:04:42.911 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:05:12.913 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:05:12.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:05:15.970 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:05:15.971 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:05:45.973 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:05:45.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:05:49.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:05:49.054 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:06:19.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:06:19.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:06:22.110 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:06:22.110 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:06:52.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:06:52.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:06:55.170 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:06:55.171 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:07:25.172 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:07:25.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:07:28.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:07:28.255 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:07:58.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:07:58.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:08:01.310 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:08:01.311 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:08:31.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:08:31.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:08:34.366 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:08:34.366 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:09:04.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:09:04.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:09:07.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:09:07.423 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:09:37.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:09:37.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:09:40.478 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:09:40.479 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:10:10.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:10:10.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:10:13.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:10:13.536 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:10:43.537 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:10:43.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:10:46.594 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:10:46.595 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:11:16.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:11:16.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:11:19.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:11:19.679 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:11:49.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:11:49.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:11:52.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:11:52.734 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:12:22.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:12:22.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:12:25.790 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:12:25.790 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:12:55.792 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:12:55.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:12:58.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:12:58.847 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:13:28.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:13:28.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:13:31.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:13:31.903 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:14:01.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:14:01.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:14:04.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:14:04.959 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:14:34.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:14:34.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:14:38.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:14:38.015 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:15:08.017 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:15:08.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:15:11.072 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:15:11.072 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:15:41.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:15:41.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:15:44.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:15:44.159 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:16:14.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:16:14.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:16:17.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:16:17.215 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:16:47.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:16:47.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:16:50.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:16:50.270 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:17:20.272 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:17:20.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:17:23.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:17:23.326 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:17:53.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:17:53.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:17:56.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:17:56.382 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:18:26.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:18:26.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:18:29.442 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:18:29.443 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:18:59.444 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:18:59.453 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:19:02.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:19:02.526 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:19:32.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:19:32.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:19:35.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:19:35.582 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:20:05.583 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:20:05.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:20:08.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:20:08.639 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:20:38.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:20:38.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:20:41.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:20:41.694 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:21:11.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:21:11.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:21:14.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:21:14.771 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:21:44.773 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:21:44.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:21:47.842 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:21:47.843 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:22:17.844 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:22:17.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:22:20.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:22:20.926 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:22:50.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:22:50.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:22:53.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:22:53.983 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:23:23.985 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:23:23.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:23:27.074 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:23:27.075 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:23:57.077 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:23:57.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:24:00.162 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:24:00.163 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:24:30.165 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:24:30.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:24:33.250 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:24:33.250 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:25:03.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:25:03.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:25:06.338 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:25:06.339 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:25:36.341 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:25:36.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:25:39.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:25:39.422 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:26:09.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:26:09.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:26:12.478 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:26:12.479 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:26:42.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:26:42.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:26:45.538 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:26:45.538 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:27:15.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:27:15.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:27:18.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:27:18.623 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:27:48.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:27:48.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:27:51.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:27:51.711 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:28:21.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:28:21.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:28:24.770 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:28:24.772 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:28:54.773 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:28:54.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:28:57.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:28:57.855 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:29:27.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:29:27.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:29:30.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:29:30.911 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:30:00.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:30:00.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:30:03.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:30:03.967 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:30:33.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:30:33.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:30:37.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:30:37.022 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:31:07.024 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:31:07.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:31:10.078 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:31:10.078 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:31:40.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:31:40.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:31:43.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:31:43.134 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:32:13.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:32:13.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:32:16.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:32:16.191 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:32:46.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:32:46.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:32:49.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:32:49.247 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:33:19.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:33:19.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:33:22.302 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:33:22.303 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:33:52.305 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:33:52.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:33:55.362 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:33:55.363 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:34:25.364 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:34:25.366 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:34:28.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:34:28.447 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:34:58.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:34:58.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:35:01.502 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:35:01.502 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:35:31.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:35:31.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:35:34.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:35:34.558 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:36:04.560 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:36:04.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:36:07.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:36:07.647 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:36:37.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:36:37.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:36:40.702 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:36:40.703 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:37:10.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:37:10.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:37:13.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:37:13.759 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:37:43.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:37:43.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:37:46.814 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:37:46.815 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:38:16.817 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:38:16.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:38:19.870 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:38:19.871 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:38:49.873 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:38:49.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:38:52.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:38:52.927 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:39:22.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:39:22.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:39:25.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:39:25.983 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:39:55.983 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:39:55.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:39:59.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:39:59.039 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:40:29.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:40:29.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:40:32.094 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:40:32.095 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:41:02.096 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:41:02.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:41:05.150 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:41:05.150 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:41:35.152 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:41:35.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:41:38.210 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:41:38.211 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:42:08.212 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:42:08.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:42:11.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:42:11.295 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:42:41.296 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:42:41.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:42:44.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:42:44.351 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:43:14.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:43:14.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:43:17.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:43:17.406 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:43:47.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:43:47.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:43:50.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:43:50.463 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:44:20.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:44:20.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:44:23.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:44:23.519 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:44:53.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:44:53.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:44:56.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:44:56.575 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:45:26.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:45:26.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:45:29.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:45:29.631 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:45:59.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:45:59.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:46:02.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:46:02.686 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:46:32.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:46:32.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:46:35.743 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:46:35.745 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:47:05.747 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:47:05.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:47:08.834 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:47:08.835 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:47:38.836 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:47:38.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:47:41.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:47:41.919 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:48:11.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:48:11.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:48:14.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:48:14.975 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:48:44.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:48:44.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:48:48.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:48:48.030 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:49:18.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:49:18.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:49:21.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:49:21.086 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:49:51.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:49:51.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:49:54.146 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:49:54.146 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:50:24.148 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:50:24.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:50:27.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:50:27.230 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:50:57.231 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:50:57.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:51:00.286 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:51:00.286 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:51:30.288 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:51:30.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:51:33.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:51:33.342 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:52:03.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:52:03.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:52:06.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:52:06.398 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:52:36.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:52:36.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:52:39.458 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:52:39.459 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:53:09.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:53:09.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:53:12.546 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:53:12.547 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:53:42.548 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:53:42.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:53:45.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:53:45.631 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:54:15.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:54:15.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:54:18.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:54:18.687 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:54:48.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:54:48.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:54:51.746 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:54:51.747 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:55:21.749 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:55:21.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:55:24.834 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:55:24.835 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:55:54.836 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:55:54.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:55:57.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:55:57.918 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:56:27.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:56:27.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:56:30.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:56:30.974 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:57:00.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:57:00.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:57:04.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:57:04.031 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:57:34.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:57:34.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:57:37.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:57:37.086 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:58:07.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:58:07.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:58:10.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:58:10.143 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:58:40.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:58:40.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:58:43.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:58:43.199 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:59:13.200 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:59:13.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:59:16.262 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:59:16.263 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-18T23:59:46.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-18T23:59:46.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-18T23:59:49.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-18T23:59:49.342 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:00:19.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:00:19.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:00:22.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:00:22.399 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:00:52.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:00:52.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:00:55.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:00:55.454 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:01:25.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:01:25.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:01:28.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:01:28.511 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:01:58.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:01:58.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:02:01.566 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:02:01.566 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:02:31.568 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:02:31.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:02:34.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:02:34.622 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:03:04.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:03:04.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:03:07.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:03:07.679 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:03:37.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:03:37.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:03:40.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:03:40.734 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:04:10.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:04:10.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:04:13.790 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:04:13.791 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:04:43.792 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:04:43.793 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:04:46.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:04:46.847 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:05:16.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:05:16.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:05:19.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:05:19.902 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:05:49.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:05:49.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:05:52.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:05:52.959 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:06:22.960 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:06:22.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:06:26.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:06:26.015 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:06:56.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:06:56.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:06:59.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:06:59.070 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:07:29.072 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:07:29.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:07:32.126 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:07:32.127 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:08:02.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:08:02.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:08:05.182 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:08:05.183 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:08:35.184 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:08:35.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:08:38.238 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:08:38.239 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:09:08.240 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:09:08.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:09:11.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:09:11.294 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:09:41.296 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:09:41.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:09:44.354 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:09:44.355 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:10:14.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:10:14.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:10:17.442 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:10:17.442 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:10:47.444 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:10:47.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:10:50.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:10:50.527 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:11:20.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:11:20.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:11:23.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:11:23.583 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:11:53.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:11:53.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:11:56.642 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:11:56.643 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:12:26.645 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:12:26.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:12:29.730 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:12:29.731 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:12:59.733 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:12:59.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:13:02.814 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:13:02.814 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:13:32.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:13:32.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:13:35.870 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:13:35.871 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:14:05.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:14:05.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:14:08.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:14:08.926 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:14:38.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:14:38.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:14:41.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:14:41.983 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:15:11.984 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:15:11.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:15:15.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:15:15.039 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:15:45.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:15:45.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:15:48.094 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:15:48.095 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:16:18.095 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:16:18.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:16:21.154 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:16:21.155 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:16:51.157 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:16:51.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:16:54.238 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:16:54.238 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:17:24.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:17:24.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:17:27.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:17:27.326 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:17:57.327 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:17:57.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:18:00.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:18:00.383 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:18:30.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:18:30.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:18:33.438 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:18:33.439 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:19:03.440 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:19:03.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:19:06.494 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:19:06.495 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:19:36.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:19:36.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:19:39.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:19:39.551 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:20:09.552 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:20:09.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:20:12.606 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:20:12.606 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:20:42.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:20:42.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:20:45.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:20:45.663 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:21:15.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:21:15.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:21:18.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:21:18.718 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:21:48.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:21:48.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:21:51.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:21:51.775 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:22:21.779 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:22:21.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:22:24.866 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:22:24.866 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:22:54.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:22:54.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:22:57.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:22:57.951 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:23:27.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:23:27.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:23:31.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:23:31.007 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:24:01.007 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:24:01.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:24:04.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:24:04.063 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:24:34.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:24:34.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:24:37.122 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:24:37.123 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:25:07.124 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:25:07.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:25:10.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:25:10.206 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:25:40.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:25:40.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:25:43.262 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:25:43.263 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:26:13.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:26:13.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:26:16.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:26:16.319 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:26:46.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:26:46.322 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:26:49.374 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:26:49.375 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:27:19.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:27:19.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:27:22.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:27:22.430 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:27:52.432 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:27:52.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:27:55.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:27:55.487 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:28:25.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:28:25.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:28:28.542 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:28:28.542 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:28:58.544 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:28:58.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:29:01.598 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:29:01.598 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:29:31.599 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:29:31.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:29:34.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:29:34.654 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:30:04.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:30:04.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:30:07.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:30:07.710 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:30:37.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:30:37.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:30:40.770 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:30:40.771 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:31:10.772 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:31:10.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:31:13.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:31:13.855 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:31:43.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:31:43.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:31:46.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:31:46.911 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:32:16.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:32:16.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:32:19.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:32:19.967 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:32:49.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:32:49.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:32:53.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:32:53.022 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:33:23.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:33:23.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:33:26.082 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:33:26.083 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:33:56.084 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:33:56.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:33:59.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:33:59.167 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:34:29.172 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:34:29.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:34:32.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:34:32.255 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:35:02.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:35:02.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:35:05.310 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:35:05.311 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:35:35.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:35:35.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:35:38.366 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:35:38.366 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:36:08.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:36:08.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:36:11.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:36:11.422 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:36:41.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:36:41.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:36:44.478 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:36:44.478 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:37:14.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:37:14.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:37:17.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:37:17.534 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:37:47.535 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:37:47.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:37:50.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:37:50.590 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:38:20.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:38:20.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:38:23.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:38:23.646 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:38:53.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:38:53.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:38:56.702 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:38:56.703 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:39:26.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:39:26.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:39:29.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:39:29.759 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:39:59.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:39:59.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:40:02.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:40:02.847 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:40:32.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:40:32.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:40:35.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:40:35.902 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:41:05.903 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:41:05.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:41:08.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:41:08.958 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:41:38.960 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:41:38.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:41:42.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:41:42.014 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:42:12.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:42:12.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:42:15.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:42:15.071 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:42:45.072 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:42:45.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:42:48.126 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:42:48.126 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:43:18.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:43:18.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:43:21.182 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:43:21.183 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:43:51.184 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:43:51.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:43:54.242 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:43:54.243 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:44:24.244 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:44:24.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:44:27.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:44:27.326 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:44:57.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:44:57.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:45:00.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:45:00.382 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:45:30.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:45:30.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:45:33.438 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:45:33.438 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:46:03.440 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:46:03.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:46:06.494 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:46:06.495 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:46:36.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:46:36.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:46:39.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:46:39.550 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:47:09.552 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:47:09.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:47:12.606 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:47:12.606 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:47:42.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:47:42.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:47:45.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:47:45.663 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:48:15.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:48:15.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:48:18.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:48:18.718 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:48:48.722 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:48:48.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:48:51.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:48:51.774 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:49:21.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:49:21.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:49:24.830 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:49:24.831 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:49:54.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:49:54.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:49:57.886 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:49:57.886 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:50:27.888 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:50:27.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:50:30.942 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:50:30.943 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:51:00.944 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:51:00.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:51:04.000 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:51:04.001 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:51:34.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:51:34.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:51:37.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:51:37.054 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:52:07.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:52:07.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:52:10.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:52:10.143 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:52:40.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:52:40.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:52:43.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:52:43.198 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:53:13.200 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:53:13.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:53:16.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:53:16.254 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:53:46.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:53:46.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:53:49.310 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:53:49.310 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:54:19.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:54:19.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:54:22.366 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:54:22.367 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:54:52.369 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:54:52.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:54:55.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:54:55.455 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:55:25.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:55:25.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:55:28.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:55:28.511 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:55:58.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:55:58.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:56:01.566 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:56:01.566 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:56:31.568 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:56:31.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:56:34.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:56:34.623 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:57:04.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:57:04.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:57:07.687 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:57:07.688 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:57:37.689 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:57:37.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:57:40.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:57:40.767 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:58:10.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:58:10.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:58:13.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:58:13.855 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:58:43.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:58:43.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:58:46.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:58:46.911 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:59:16.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:59:16.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:59:19.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:59:19.966 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T00:59:49.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T00:59:49.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T00:59:53.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T00:59:53.023 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:00:23.025 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:00:23.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:00:26.078 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:00:26.078 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:00:56.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:00:56.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:00:59.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:00:59.134 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:01:29.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:01:29.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:01:32.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:01:32.190 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:02:02.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:02:02.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:02:05.250 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:02:05.251 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:02:35.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:02:35.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:02:38.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:02:38.334 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:03:08.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:03:08.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:03:11.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:03:11.390 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:03:41.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:03:41.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:03:44.450 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:03:44.450 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:04:14.452 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:04:14.453 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:04:17.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:04:17.535 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:04:47.537 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:04:47.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:04:50.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:04:50.591 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:05:20.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:05:20.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:05:23.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:05:23.646 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:05:53.647 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:05:53.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:05:56.706 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:05:56.707 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:06:26.707 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:06:26.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:06:29.790 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:06:29.790 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:06:59.792 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:06:59.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:07:02.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:07:02.846 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:07:32.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:07:32.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:07:35.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:07:35.902 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:08:05.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:08:05.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:08:08.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:08:08.958 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:08:38.960 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:08:38.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:08:42.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:08:42.015 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:09:12.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:09:12.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:09:15.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:09:15.071 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:09:45.072 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:09:45.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:09:48.126 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:09:48.126 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:10:18.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:10:18.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:10:21.182 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:10:21.183 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:10:51.184 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:10:51.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:10:54.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:10:54.271 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:11:24.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:11:24.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:11:27.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:11:27.326 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:11:57.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:11:57.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:12:00.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:12:00.383 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:12:30.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:12:30.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:12:33.438 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:12:33.439 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:13:03.440 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:13:03.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:13:06.498 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:13:06.498 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:13:36.500 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:13:36.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:13:39.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:13:39.583 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:14:09.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:14:09.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:14:12.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:14:12.638 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:14:42.639 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:14:42.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:14:45.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:14:45.695 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:15:15.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:15:15.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:15:18.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:15:18.750 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:15:48.751 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:15:48.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:15:51.806 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:15:51.806 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:16:21.807 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:16:21.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:16:24.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:16:24.862 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:16:54.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:16:54.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:16:57.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:16:57.919 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:17:27.921 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:17:27.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:17:30.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:17:30.975 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:18:00.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:18:00.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:18:04.034 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:18:04.035 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:18:34.036 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:18:34.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:18:37.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:18:37.119 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:19:07.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:19:07.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:19:10.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:19:10.206 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:19:40.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:19:40.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:19:43.264 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:19:43.265 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:20:13.266 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:20:13.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:20:16.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:20:16.350 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:20:46.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:20:46.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:20:49.407 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:20:49.408 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:21:19.409 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:21:19.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:21:22.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:21:22.463 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:21:52.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:21:52.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:21:55.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:21:55.518 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:22:25.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:22:25.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:22:28.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:22:28.575 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:22:58.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:22:58.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:23:01.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:23:01.631 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:23:31.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:23:31.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:23:34.690 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:23:34.691 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:24:04.692 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:24:04.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:24:07.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:24:07.775 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:24:37.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:24:37.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:24:40.830 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:24:40.831 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:25:10.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:25:10.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:25:13.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:25:13.895 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:25:43.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:25:43.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:25:46.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:25:46.974 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:26:16.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:26:16.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:26:20.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:26:20.031 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:26:50.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:26:50.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:26:53.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:26:53.087 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:27:23.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:27:23.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:27:26.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:27:26.142 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:27:56.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:27:56.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:27:59.202 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:27:59.202 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:28:29.204 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:28:29.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:28:32.290 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:28:32.290 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:29:02.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:29:02.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:29:05.374 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:29:05.375 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:29:35.377 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:29:35.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:29:38.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:29:38.431 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:30:08.433 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:30:08.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:30:11.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:30:11.519 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:30:41.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:30:41.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:30:44.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:30:44.575 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:31:14.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:31:14.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:31:17.633 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:31:17.634 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:31:47.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:31:47.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:31:50.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:31:50.719 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:32:20.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:32:20.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:32:23.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:32:23.775 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:32:53.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:32:53.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:32:56.834 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:32:56.835 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:33:26.836 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:33:26.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:33:29.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:33:29.919 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:33:59.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:33:59.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:34:03.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:34:03.006 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:34:33.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:34:33.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:34:36.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:34:36.062 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:35:06.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:35:06.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:35:09.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:35:09.119 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:35:39.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:35:39.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:35:42.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:35:42.175 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:36:12.177 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:36:12.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:36:15.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:36:15.231 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:36:45.233 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:36:45.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:36:48.290 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:36:48.290 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:37:18.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:37:18.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:37:21.378 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:37:21.379 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:37:51.380 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:37:51.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:37:54.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:37:54.462 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:38:24.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:38:24.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:38:27.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:38:27.518 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:38:57.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:38:57.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:39:00.610 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:39:00.611 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:39:30.612 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:39:30.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:39:33.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:39:33.694 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:40:03.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:40:03.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:40:06.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:40:06.751 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:40:36.758 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:40:36.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:40:39.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:40:39.839 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:41:09.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:41:09.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:41:12.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:41:12.894 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:41:42.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:41:42.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:41:45.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:41:45.951 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:42:15.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:42:15.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:42:19.010 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:42:19.011 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:42:49.012 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:42:49.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:42:52.098 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:42:52.099 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:43:22.100 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:43:22.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:43:25.186 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:43:25.186 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:43:55.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:43:55.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:43:58.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:43:58.271 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:44:28.271 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:44:28.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:44:31.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:44:31.327 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:45:01.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:45:01.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:45:04.386 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:45:04.387 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:45:34.388 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:45:34.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:45:37.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:45:37.471 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:46:07.472 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:46:07.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:46:10.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:46:10.527 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:46:40.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:46:40.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:46:43.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:46:43.583 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:47:13.585 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:47:13.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:47:16.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:47:16.639 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:47:46.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:47:46.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:47:49.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:47:49.695 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:48:19.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:48:19.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:48:22.782 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:48:22.783 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:48:52.784 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:48:52.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:48:55.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:48:55.839 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:49:25.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:49:25.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:49:28.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:49:28.895 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:49:58.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:49:58.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:50:01.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:50:01.951 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:50:31.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:50:31.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:50:35.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:50:35.007 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:51:05.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:51:05.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:51:08.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:51:08.063 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:51:38.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:51:38.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:51:41.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:51:41.118 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:52:11.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:52:11.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:52:14.178 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:52:14.179 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:52:44.181 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:52:44.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:52:47.262 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:52:47.263 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:53:17.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:53:17.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:53:20.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:53:20.319 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:53:50.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:53:50.322 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:53:53.374 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:53:53.374 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:54:23.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:54:23.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:54:26.434 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:54:26.435 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:54:56.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:54:56.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:54:59.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:54:59.519 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:55:29.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:55:29.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:55:32.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:55:32.575 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:56:02.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:56:02.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:56:05.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:56:05.663 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:56:35.665 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:56:35.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:56:38.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:56:38.719 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:57:08.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:57:08.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:57:11.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:57:11.775 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:57:41.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:57:41.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:57:44.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:57:44.862 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:58:14.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:58:14.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:58:17.922 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:58:17.923 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:58:47.924 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:58:47.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:58:51.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:58:51.007 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:59:21.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:59:21.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:59:24.066 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:59:24.067 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T01:59:54.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T01:59:54.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T01:59:57.150 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T01:59:57.151 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:00:27.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:00:27.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:00:30.213 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:00:30.214 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:01:00.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:01:00.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:01:03.298 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:01:03.299 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:01:33.300 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:01:33.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:01:36.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:01:36.383 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:02:06.385 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:02:06.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:02:09.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:02:09.471 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:02:39.472 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:02:39.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:02:42.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:02:42.559 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:03:12.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:03:12.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:03:15.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:03:15.615 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:03:45.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:03:45.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:03:48.670 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:03:48.671 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:04:18.675 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:04:18.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:04:21.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:04:21.759 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:04:51.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:04:51.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:04:54.819 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:04:54.819 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:05:24.821 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:05:24.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:05:27.906 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:05:27.906 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:05:57.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:05:57.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:06:00.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:06:00.991 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:06:30.993 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:06:30.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:06:34.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:06:34.047 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:07:04.050 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:07:04.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:07:07.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:07:07.135 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:07:37.138 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:07:37.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:07:40.226 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:07:40.227 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:08:10.228 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:08:10.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:08:13.311 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:08:13.312 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:08:43.314 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:08:43.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:08:46.403 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:08:46.404 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:09:16.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:09:16.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:09:19.490 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:09:19.491 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:09:49.493 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:09:49.495 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:09:52.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:09:52.575 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:10:22.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:10:22.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:10:25.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:10:25.631 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:10:55.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:10:55.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:10:58.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:10:58.718 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:11:28.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:11:28.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:11:31.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:11:31.775 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:12:01.777 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:12:01.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:12:04.830 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:12:04.830 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:12:34.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:12:34.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:12:37.886 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:12:37.887 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:13:07.889 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:13:07.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:13:10.942 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:13:10.943 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:13:40.944 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:13:40.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:13:43.998 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:13:43.999 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:14:14.000 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:14:14.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:14:17.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:14:17.055 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:14:47.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:14:47.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:14:50.110 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:14:50.111 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:15:20.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:15:20.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:15:23.172 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:15:23.173 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:15:53.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:15:53.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:15:56.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:15:56.255 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:16:26.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:16:26.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:16:29.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:16:29.343 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:16:59.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:16:59.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:17:02.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:17:02.399 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:17:32.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:17:32.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:17:35.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:17:35.455 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:18:05.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:18:05.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:18:08.511 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:18:08.512 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:18:38.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:18:38.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:18:41.598 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:18:41.599 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:19:11.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:19:11.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:19:14.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:19:14.655 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:19:44.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:19:44.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:19:47.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:19:47.710 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:20:17.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:20:17.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:20:20.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:20:20.766 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:20:50.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:20:50.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:20:53.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:20:53.855 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:21:23.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:21:23.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:21:26.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:21:26.911 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:21:56.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:21:56.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:21:59.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:21:59.967 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:22:29.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:22:29.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:22:33.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:22:33.023 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:23:03.025 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:23:03.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:23:06.078 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:23:06.079 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:23:36.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:23:36.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:23:39.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:23:39.135 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:24:09.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:24:09.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:24:12.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:24:12.190 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:24:42.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:24:42.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:24:45.250 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:24:45.251 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:25:15.253 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:25:15.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:25:18.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:25:18.334 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:25:48.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:25:48.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:25:51.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:25:51.391 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:26:21.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:26:21.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:26:24.450 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:26:24.451 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:26:54.452 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:26:54.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:26:57.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:26:57.535 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:27:27.536 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:27:27.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:27:30.591 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:27:30.592 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:28:00.594 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:28:00.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:28:03.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:28:03.679 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:28:33.681 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:28:33.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:28:36.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:28:36.735 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:29:06.737 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:29:06.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:29:09.822 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:29:09.823 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:29:39.824 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:29:39.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:29:42.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:29:42.911 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:30:12.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:30:12.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:30:15.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:30:15.967 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:30:45.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:30:45.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:30:49.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:30:49.022 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:31:19.024 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:31:19.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:31:22.078 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:31:22.079 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:31:52.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:31:52.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:31:55.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:31:55.134 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:32:25.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:32:25.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:32:28.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:32:28.191 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:32:58.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:32:58.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:33:01.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:33:01.247 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:33:31.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:33:31.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:33:34.302 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:33:34.303 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:34:04.304 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:34:04.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:34:07.358 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:34:07.359 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:34:37.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:34:37.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:34:40.414 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:34:40.415 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:35:10.419 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:35:10.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:35:13.502 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:35:13.503 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:35:43.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:35:43.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:35:46.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:35:46.559 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:36:16.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:36:16.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:36:19.618 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:36:19.619 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:36:49.620 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:36:49.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:36:52.702 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:36:52.703 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:37:22.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:37:22.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:37:25.762 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:37:25.763 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:37:55.764 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:37:55.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:37:58.850 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:37:58.851 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:38:28.852 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:38:28.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:38:31.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:38:31.935 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:39:01.936 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:39:01.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:39:04.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:39:04.991 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:39:34.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:39:34.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:39:38.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:39:38.047 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:40:08.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:40:08.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:40:11.106 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:40:11.107 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:40:41.109 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:40:41.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:40:44.194 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:40:44.195 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:41:14.196 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:41:14.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:41:17.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:41:17.323 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:41:47.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:41:47.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:41:50.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:41:50.399 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:42:20.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:42:20.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:42:23.458 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:42:23.459 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:42:53.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:42:53.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:42:56.542 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:42:56.543 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:43:26.544 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:43:26.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:43:29.598 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:43:29.599 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:43:59.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:43:59.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:44:02.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:44:02.655 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:44:32.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:44:32.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:44:35.715 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:44:35.715 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:45:05.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:45:05.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:45:08.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:45:08.799 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:45:38.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:45:38.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:45:41.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:45:41.855 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:46:11.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:46:11.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:46:14.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:46:14.911 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:46:44.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:46:44.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:46:47.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:46:47.967 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:47:17.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:47:17.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:47:21.027 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:47:21.028 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:47:51.029 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:47:51.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:47:54.110 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:47:54.111 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:48:24.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:48:24.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:48:27.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:48:27.167 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:48:57.168 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:48:57.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:49:00.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:49:00.223 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:49:30.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:49:30.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:49:33.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:49:33.279 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:50:03.281 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:50:03.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:50:06.366 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:50:06.367 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:50:36.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:50:36.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:50:39.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:50:39.455 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:51:09.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:51:09.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:51:12.514 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:51:12.515 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:51:42.517 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:51:42.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:51:45.602 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:51:45.603 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:52:15.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:52:15.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:52:18.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:52:18.687 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:52:48.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:52:48.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:52:51.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:52:51.743 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:53:21.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:53:21.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:53:24.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:53:24.799 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:53:54.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:53:54.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:53:57.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:53:57.855 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:54:27.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:54:27.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:54:30.914 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:54:30.915 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:55:00.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:55:00.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:55:03.998 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:55:03.999 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:55:34.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:55:34.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:55:37.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:55:37.054 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:56:07.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:56:07.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:56:10.110 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:56:10.111 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:56:40.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:56:40.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:56:43.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:56:43.167 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:57:13.169 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:57:13.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:57:16.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:57:16.223 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:57:46.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:57:46.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:57:49.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:57:49.279 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:58:19.281 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:58:19.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:58:22.338 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:58:22.339 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:58:52.340 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:58:52.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:58:55.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:58:55.423 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:59:25.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:59:25.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T02:59:28.478 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T02:59:28.479 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T02:59:58.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T02:59:58.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:00:01.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:00:01.535 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:00:31.537 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:00:31.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:00:34.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:00:34.590 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:01:04.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:01:04.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:01:07.650 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:01:07.650 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:01:37.652 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:01:37.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:01:40.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:01:40.735 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:02:10.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:02:10.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:02:13.790 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:02:13.791 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:02:43.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:02:43.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:02:46.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:02:46.847 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:03:16.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:03:16.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:03:19.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:03:19.902 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:03:49.903 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:03:49.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:03:52.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:03:52.959 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:04:22.960 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:04:22.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:04:26.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:04:26.014 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:04:56.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:04:56.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:04:59.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:04:59.070 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:05:29.072 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:05:29.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:05:32.126 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:05:32.127 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:06:02.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:06:02.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:06:05.182 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:06:05.182 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:06:35.184 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:06:35.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:06:38.238 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:06:38.238 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:07:08.240 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:07:08.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:07:11.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:07:11.295 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:07:41.297 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:07:41.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:07:44.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:07:44.351 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:08:14.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:08:14.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:08:17.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:08:17.407 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:08:47.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:08:47.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:08:50.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:08:50.463 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:09:20.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:09:20.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:09:23.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:09:23.519 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:09:53.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:09:53.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:09:56.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:09:56.575 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:10:26.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:10:26.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:10:29.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:10:29.631 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:10:59.631 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:10:59.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:11:02.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:11:02.686 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:11:32.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:11:32.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:11:35.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:11:35.743 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:12:05.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:12:05.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:12:08.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:12:08.798 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:12:38.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:12:38.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:12:41.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:12:41.855 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:13:11.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:13:11.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:13:14.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:13:14.911 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:13:44.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:13:44.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:13:47.970 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:13:47.971 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:14:17.972 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:14:17.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:14:21.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:14:21.055 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:14:51.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:14:51.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:14:54.110 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:14:54.110 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:15:24.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:15:24.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:15:27.170 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:15:27.170 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:15:57.172 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:15:57.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:16:00.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:16:00.254 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:16:30.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:16:30.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:16:33.310 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:16:33.310 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:17:03.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:17:03.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:17:06.366 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:17:06.366 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:17:36.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:17:36.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:17:39.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:17:39.422 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:18:09.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:18:09.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:18:12.478 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:18:12.479 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:18:42.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:18:42.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:18:45.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:18:45.535 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:19:15.536 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:19:15.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:19:18.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:19:18.591 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:19:48.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:19:48.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:19:51.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:19:51.647 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:20:21.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:20:21.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:20:24.702 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:20:24.703 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:20:54.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:20:54.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:20:57.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:20:57.759 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:21:27.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:21:27.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:21:30.814 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:21:30.815 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:22:00.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:22:00.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:22:03.870 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:22:03.870 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:22:33.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:22:33.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:22:36.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:22:36.927 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:23:06.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:23:06.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:23:09.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:23:09.982 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:23:39.984 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:23:39.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:23:43.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:23:43.039 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:24:13.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:24:13.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:24:16.098 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:24:16.098 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:24:46.100 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:24:46.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:24:49.187 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:24:49.188 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:25:19.190 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:25:19.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:25:22.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:25:22.271 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:25:52.272 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:25:52.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:25:55.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:25:55.327 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:26:25.329 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:26:25.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:26:28.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:26:28.383 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:26:58.385 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:26:58.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:27:01.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:27:01.471 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:27:31.472 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:27:31.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:27:34.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:27:34.558 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:28:04.560 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:28:04.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:28:07.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:28:07.615 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:28:37.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:28:37.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:28:40.670 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:28:40.671 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:29:10.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:29:10.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:29:13.726 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:29:13.726 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:29:43.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:29:43.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:29:46.782 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:29:46.782 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:30:16.784 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:30:16.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:30:19.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:30:19.839 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:30:49.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:30:49.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:30:52.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:30:52.894 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:31:22.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:31:22.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:31:25.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:31:25.950 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:31:55.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:31:55.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:31:59.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:31:59.007 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:32:29.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:32:29.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:32:32.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:32:32.062 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:33:02.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:33:02.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:33:05.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:33:05.119 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:33:35.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:33:35.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:33:38.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:33:38.174 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:34:08.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:34:08.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:34:11.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:34:11.231 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:34:41.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:34:41.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:34:44.286 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:34:44.287 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:35:14.288 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:35:14.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:35:17.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:35:17.342 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:35:47.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:35:47.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:35:50.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:35:50.398 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:36:20.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:36:20.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:36:23.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:36:23.454 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:36:53.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:36:53.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:36:56.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:36:56.510 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:37:26.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:37:26.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:37:29.566 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:37:29.567 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:37:59.568 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:37:59.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:38:02.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:38:02.623 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:38:32.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:38:32.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:38:35.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:38:35.679 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:39:05.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:39:05.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:39:08.738 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:39:08.738 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:39:38.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:39:38.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:39:41.822 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:39:41.823 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:40:11.824 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:40:11.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:40:14.878 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:40:14.879 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:40:44.880 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:40:44.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:40:47.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:40:47.935 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:41:17.935 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:41:17.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:41:20.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:41:20.990 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:41:50.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:41:50.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:41:54.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:41:54.046 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:42:24.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:42:24.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:42:27.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:42:27.103 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:42:57.105 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:42:57.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:43:00.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:43:00.159 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:43:30.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:43:30.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:43:33.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:43:33.214 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:44:03.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:44:03.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:44:06.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:44:06.271 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:44:36.272 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:44:36.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:44:39.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:44:39.326 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:45:09.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:45:09.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:45:12.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:45:12.383 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:45:42.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:45:42.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:45:45.438 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:45:45.439 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:46:15.440 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:46:15.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:46:18.494 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:46:18.495 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:46:48.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:46:48.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:46:51.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:46:51.550 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:47:21.552 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:47:21.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:47:24.606 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:47:24.606 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:47:54.607 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:47:54.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:47:57.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:47:57.662 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:48:27.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:48:27.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:48:30.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:48:30.718 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:49:00.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:49:00.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:49:03.778 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:49:03.779 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:49:33.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:49:33.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:49:36.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:49:36.912 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:50:06.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:50:06.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:50:09.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:50:09.983 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:50:39.984 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:50:39.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:50:43.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:50:43.038 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:51:13.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:51:13.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:51:16.094 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:51:16.094 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:51:46.095 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:51:46.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:51:49.150 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:51:49.151 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:52:19.152 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:52:19.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:52:22.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:52:22.207 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:52:52.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:52:52.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:52:55.262 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:52:55.262 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:53:25.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:53:25.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:53:28.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:53:28.318 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:53:58.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:53:58.322 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:54:01.374 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:54:01.375 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:54:31.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:54:31.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:54:34.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:54:34.431 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:55:04.432 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:55:04.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:55:07.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:55:07.486 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:55:37.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:55:37.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:55:40.542 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:55:40.542 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:56:10.544 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:56:10.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:56:13.598 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:56:13.598 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:56:43.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:56:43.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:56:46.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:56:46.654 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:57:16.655 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:57:16.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:57:19.714 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:57:19.715 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:57:49.715 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:57:49.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:57:52.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:57:52.798 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:58:22.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:58:22.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:58:25.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:58:25.855 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:58:55.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:58:55.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:58:58.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:58:58.910 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T03:59:28.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T03:59:28.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T03:59:31.998 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T03:59:31.999 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:00:02.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:00:02.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:00:05.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:00:05.055 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:00:35.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:00:35.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:00:38.110 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:00:38.110 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:01:08.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:01:08.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:01:11.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:01:11.166 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:01:41.167 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:01:41.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:01:44.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:01:44.222 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:02:14.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:02:14.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:02:17.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:02:17.278 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:02:47.280 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:02:47.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:02:50.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:02:50.334 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:03:20.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:03:20.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:03:23.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:03:23.391 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:03:53.393 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:03:53.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:03:56.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:03:56.446 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:04:26.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:04:26.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:04:29.502 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:04:29.502 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:04:59.503 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:04:59.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:05:02.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:05:02.559 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:05:32.560 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:05:32.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:05:35.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:05:35.615 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:06:05.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:06:05.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:06:08.670 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:06:08.670 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:06:38.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:06:38.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:06:41.726 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:06:41.727 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:07:11.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:07:11.730 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:07:14.782 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:07:14.782 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:07:44.783 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:07:44.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:07:47.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:07:47.839 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:08:17.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:08:17.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:08:20.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:08:20.894 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:08:50.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:08:50.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:08:53.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:08:53.950 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:09:23.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:09:23.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:09:27.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:09:27.015 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:09:57.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:09:57.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:10:00.094 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:10:00.095 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:10:30.096 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:10:30.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:10:33.150 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:10:33.150 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:11:03.151 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:11:03.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:11:06.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:11:06.207 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:11:36.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:11:36.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:11:39.262 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:11:39.263 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:12:09.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:12:09.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:12:12.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:12:12.318 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:12:42.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:12:42.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:12:45.374 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:12:45.375 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:13:15.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:13:15.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:13:18.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:13:18.431 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:13:48.432 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:13:48.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:13:51.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:13:51.486 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:14:21.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:14:21.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:14:24.542 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:14:24.542 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:14:54.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:14:54.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:14:57.598 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:14:57.598 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:15:27.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:15:27.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:15:30.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:15:30.654 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:16:00.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:16:00.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:16:03.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:16:03.711 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:16:33.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:16:33.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:16:36.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:16:36.766 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:17:06.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:17:06.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:17:09.826 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:17:09.826 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:17:39.828 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:17:39.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:17:42.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:17:42.911 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:18:12.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:18:12.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:18:15.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:18:15.967 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:18:45.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:18:45.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:18:49.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:18:49.022 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:19:19.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:19:19.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:19:22.078 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:19:22.078 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:19:52.082 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:19:52.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:19:55.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:19:55.134 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:20:25.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:20:25.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:20:28.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:20:28.190 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:20:58.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:20:58.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:21:01.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:21:01.247 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:21:31.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:21:31.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:21:34.302 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:21:34.302 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:22:04.303 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:22:04.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:22:07.358 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:22:07.358 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:22:37.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:22:37.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:22:40.414 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:22:40.415 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:23:10.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:23:10.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:23:13.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:23:13.470 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:23:43.472 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:23:43.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:23:46.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:23:46.527 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:24:16.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:24:16.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:24:19.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:24:19.583 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:24:49.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:24:49.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:24:52.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:24:52.638 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:25:22.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:25:22.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:25:25.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:25:25.694 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:25:55.695 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:25:55.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:25:58.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:25:58.751 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:26:28.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:26:28.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:26:31.810 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:26:31.810 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:27:01.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:27:01.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:27:04.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:27:04.894 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:27:34.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:27:34.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:27:37.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:27:37.950 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:28:07.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:28:07.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:28:11.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:28:11.006 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:28:41.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:28:41.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:28:44.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:28:44.062 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:29:14.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:29:14.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:29:17.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:29:17.119 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:29:47.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:29:47.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:29:50.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:29:50.175 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:30:20.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:30:20.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:30:23.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:30:23.231 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:30:53.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:30:53.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:30:56.286 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:30:56.286 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:31:26.287 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:31:26.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:31:29.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:31:29.343 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:31:59.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:31:59.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:32:02.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:32:02.398 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:32:32.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:32:32.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:32:35.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:32:35.455 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:33:05.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:33:05.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:33:08.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:33:08.510 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:33:38.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:33:38.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:33:41.566 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:33:41.566 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:34:11.567 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:34:11.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:34:14.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:34:14.622 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:34:44.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:34:44.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:34:47.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:34:47.679 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:35:17.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:35:17.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:35:20.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:35:20.735 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:35:50.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:35:50.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:35:53.790 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:35:53.790 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:36:23.791 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:36:23.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:36:26.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:36:26.846 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:36:56.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:36:56.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:36:59.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:36:59.903 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:37:29.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:37:29.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:37:32.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:37:32.958 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:38:02.959 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:38:02.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:38:06.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:38:06.014 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:38:36.015 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:38:36.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:38:39.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:38:39.071 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:39:09.072 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:39:09.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:39:12.126 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:39:12.126 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:39:42.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:39:42.129 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:39:45.182 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:39:45.187 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:40:15.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:40:15.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:40:18.274 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:40:18.274 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:40:48.276 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:40:48.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:40:51.358 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:40:51.358 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:41:21.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:41:21.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:41:24.416 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:41:24.417 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:41:54.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:41:54.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:41:57.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:41:57.470 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:42:27.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:42:27.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:42:30.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:42:30.526 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:43:00.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:43:00.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:43:03.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:43:03.582 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:43:33.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:43:33.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:43:36.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:43:36.639 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:44:06.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:44:06.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:44:09.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:44:09.694 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:44:39.695 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:44:39.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:44:42.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:44:42.750 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:45:12.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:45:12.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:45:15.806 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:45:15.807 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:45:45.808 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:45:45.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:45:48.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:45:48.862 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:46:18.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:46:18.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:46:21.922 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:46:21.923 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:46:51.924 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:46:51.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:46:55.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:46:55.007 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:47:25.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:47:25.009 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:47:28.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:47:28.062 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:47:58.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:47:58.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:48:01.122 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:48:01.123 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:48:31.124 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:48:31.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:48:34.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:48:34.206 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:49:04.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:49:04.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:49:07.262 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:49:07.263 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:49:37.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:49:37.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:49:40.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:49:40.318 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:50:10.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:50:10.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:50:13.374 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:50:13.375 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:50:43.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:50:43.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:50:46.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:50:46.434 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:51:16.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:51:16.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:51:19.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:51:19.519 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:51:49.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:51:49.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:51:52.578 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:51:52.578 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:52:22.580 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:52:22.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:52:25.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:52:25.662 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:52:55.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:52:55.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:52:58.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:52:58.718 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:53:28.719 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:53:28.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:53:31.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:53:31.774 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:54:01.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:54:01.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:54:04.830 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:54:04.830 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:54:34.831 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:54:34.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:54:37.886 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:54:37.887 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:55:07.889 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:55:07.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:55:10.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:55:10.974 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:55:40.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:55:40.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:55:44.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:55:44.031 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:56:14.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:56:14.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:56:17.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:56:17.086 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:56:47.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:56:47.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:56:50.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:56:50.142 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:57:20.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:57:20.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:57:23.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:57:23.199 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:57:53.200 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:57:53.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:57:56.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:57:56.255 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:58:26.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:58:26.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:58:29.310 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:58:29.311 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:58:59.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:58:59.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:59:02.366 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:59:02.367 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T04:59:32.369 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T04:59:32.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T04:59:35.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T04:59:35.423 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:00:05.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:00:05.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:00:08.478 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:00:08.479 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:00:38.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:00:38.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:00:41.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:00:41.535 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:01:11.537 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:01:11.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:01:14.594 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:01:14.595 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:01:44.597 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:01:44.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:01:47.682 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:01:47.690 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:02:17.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:02:17.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:02:20.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:02:20.767 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:02:50.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:02:50.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:02:53.822 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:02:53.823 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:03:23.824 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:03:23.825 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:03:26.878 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:03:26.878 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:03:56.879 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:03:56.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:03:59.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:03:59.934 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:04:29.936 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:04:29.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:04:32.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:04:32.990 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:05:02.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:05:02.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:05:06.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:05:06.046 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:05:36.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:05:36.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:05:39.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:05:39.103 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:06:09.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:06:09.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:06:12.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:06:12.159 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:06:42.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:06:42.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:06:45.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:06:45.214 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:07:15.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:07:15.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:07:18.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:07:18.271 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:07:48.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:07:48.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:07:51.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:07:51.327 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:08:21.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:08:21.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:08:24.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:08:24.383 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:08:54.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:08:54.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:08:57.438 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:08:57.439 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:09:27.440 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:09:27.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:09:30.494 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:09:30.494 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:10:00.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:10:00.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:10:03.554 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:10:03.554 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:10:33.555 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:10:33.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:10:36.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:10:36.639 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:11:06.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:11:06.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:11:09.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:11:09.695 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:11:39.697 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:11:39.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:11:42.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:11:42.750 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:12:12.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:12:12.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:12:15.806 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:12:15.806 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:12:45.808 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:12:45.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:12:48.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:12:48.862 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:13:18.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:13:18.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:13:21.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:13:21.918 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:13:51.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:13:51.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:13:54.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:13:54.974 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:14:24.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:14:24.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:14:28.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:14:28.030 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:14:58.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:14:58.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:15:01.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:15:01.086 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:15:31.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:15:31.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:15:34.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:15:34.143 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:16:04.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:16:04.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:16:07.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:16:07.199 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:16:37.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:16:37.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:16:40.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:16:40.254 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:17:10.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:17:10.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:17:13.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:17:13.343 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:17:43.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:17:43.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:17:46.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:17:46.399 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:18:16.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:18:16.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:18:19.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:18:19.455 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:18:49.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:18:49.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:18:52.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:18:52.510 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:19:22.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:19:22.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:19:25.566 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:19:25.567 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:19:55.568 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:19:55.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:19:58.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:19:58.622 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:20:28.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:20:28.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:20:31.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:20:31.679 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:21:01.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:21:01.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:21:04.738 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:21:04.738 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:21:34.740 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:21:34.741 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:21:37.822 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:21:37.822 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:22:07.824 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:22:07.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:22:10.878 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:22:10.878 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:22:40.880 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:22:40.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:22:43.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:22:43.934 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:23:13.936 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:23:13.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:23:16.991 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:23:16.991 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:23:46.993 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:23:46.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:23:50.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:23:50.047 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:24:20.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:24:20.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:24:23.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:24:23.102 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:24:53.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:24:53.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:24:56.163 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:24:56.163 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:25:26.165 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:25:26.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:25:29.249 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:25:29.250 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:25:59.251 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:25:59.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:26:02.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:26:02.334 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:26:32.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:26:32.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:26:35.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:26:35.391 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:27:05.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:27:05.393 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:27:08.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:27:08.446 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:27:38.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:27:38.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:27:41.502 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:27:41.502 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:28:11.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:28:11.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:28:14.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:28:14.559 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:28:44.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:28:44.561 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:28:47.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:28:47.615 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:29:17.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:29:17.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:29:20.670 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:29:20.671 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:29:50.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:29:50.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:29:53.730 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:29:53.731 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:30:23.732 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:30:23.733 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:30:26.814 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:30:26.815 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:30:56.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:30:56.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:30:59.870 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:30:59.870 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:31:29.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:31:29.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:31:32.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:31:32.926 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:32:02.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:32:02.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:32:05.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:32:05.982 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:32:35.984 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:32:35.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:32:39.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:32:39.038 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:33:09.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:33:09.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:33:12.094 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:33:12.094 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:33:42.096 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:33:42.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:33:45.150 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:33:45.151 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:34:15.152 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:34:15.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:34:18.210 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:34:18.211 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:34:48.212 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:34:48.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:34:51.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:34:51.294 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:35:21.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:35:21.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:35:24.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:35:24.351 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:35:54.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:35:54.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:35:57.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:35:57.407 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:36:27.409 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:36:27.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:36:30.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:36:30.462 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:37:00.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:37:00.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:37:03.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:37:03.551 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:37:33.552 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:37:33.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:37:36.606 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:37:36.606 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:38:06.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:38:06.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:38:09.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:38:09.662 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:38:39.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:38:39.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:38:42.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:38:42.718 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:39:12.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:39:12.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:39:15.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:39:15.775 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:39:45.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:39:45.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:39:48.834 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:39:48.834 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:40:18.836 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:40:18.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:40:21.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:40:21.919 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:40:51.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:40:51.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:40:54.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:40:54.975 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:41:24.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:41:24.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:41:28.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:41:28.031 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:41:58.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:41:58.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:42:01.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:42:01.086 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:42:31.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:42:31.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:42:34.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:42:34.143 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:43:04.143 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:43:04.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:43:07.202 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:43:07.202 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:43:37.204 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:43:37.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:43:40.286 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:43:40.286 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:44:10.288 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:44:10.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:44:13.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:44:13.343 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:44:43.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:44:43.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:44:46.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:44:46.403 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:45:16.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:45:16.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:45:19.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:45:19.487 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:45:49.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:45:49.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:45:52.542 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:45:52.542 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:46:22.544 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:46:22.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:46:25.602 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:46:25.603 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:46:55.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:46:55.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:46:58.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:46:58.687 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:47:28.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:47:28.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:47:31.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:47:31.743 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:48:01.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:48:01.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:48:04.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:48:04.799 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:48:34.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:48:34.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:48:37.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:48:37.854 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:49:07.855 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:49:07.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:49:10.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:49:10.911 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:49:40.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:49:40.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:49:43.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:49:43.967 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:50:13.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:50:13.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:50:17.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:50:17.022 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:50:47.024 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:50:47.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:50:50.110 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:50:50.111 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:51:20.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:51:20.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:51:23.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:51:23.167 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:51:53.169 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:51:53.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:51:56.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:51:56.222 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:52:26.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:52:26.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:52:29.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:52:29.278 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:52:59.279 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:52:59.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:53:02.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:53:02.342 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:53:32.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:53:32.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:53:35.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:53:35.423 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:54:05.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:54:05.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:54:08.478 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:54:08.478 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:54:38.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:54:38.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:54:41.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:54:41.534 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:55:11.536 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:55:11.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:55:14.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:55:14.591 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:55:44.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:55:44.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:55:47.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:55:47.647 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:56:17.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:56:17.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:56:20.702 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:56:20.702 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:56:50.703 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:56:50.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:56:53.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:56:53.759 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:57:23.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:57:23.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:57:26.818 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:57:26.819 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:57:56.820 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:57:56.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:57:59.906 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:57:59.907 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:58:29.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:58:29.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:58:32.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:58:32.990 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:59:02.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:59:02.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:59:06.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:59:06.046 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T05:59:36.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T05:59:36.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T05:59:39.103 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T05:59:39.103 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:00:09.105 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:00:09.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:00:12.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:00:12.159 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:00:42.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:00:42.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:00:45.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:00:45.215 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:01:15.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:01:15.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:01:18.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:01:18.271 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:01:48.272 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:01:48.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:01:51.362 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:01:51.363 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:02:21.364 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:02:21.366 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:02:24.450 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:02:24.451 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:02:54.453 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:02:54.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:02:57.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:02:57.534 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:03:27.535 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:03:27.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:03:30.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:03:30.599 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:04:00.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:04:00.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:04:03.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:04:03.679 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:04:33.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:04:33.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:04:36.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:04:36.735 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:05:06.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:05:06.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:05:09.790 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:05:09.791 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:05:39.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:05:39.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:05:42.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:05:42.847 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:06:12.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:06:12.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:06:15.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:06:15.902 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:06:45.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:06:45.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:06:48.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:06:48.959 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:07:18.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:07:18.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:07:22.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:07:22.015 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:07:52.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:07:52.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:07:55.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:07:55.103 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:08:25.105 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:08:25.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:08:28.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:08:28.159 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:08:58.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:08:58.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:09:01.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:09:01.214 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:09:31.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:09:31.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:09:34.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:09:34.271 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:10:04.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:10:04.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:10:07.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:10:07.391 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:10:37.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:10:37.397 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:10:40.478 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:10:40.479 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:11:10.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:11:10.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:11:13.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:11:13.535 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:11:43.537 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:11:43.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:11:46.594 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:11:46.595 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:12:16.596 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:12:16.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:12:19.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:12:19.679 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:12:49.681 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:12:49.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:12:52.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:12:52.767 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:13:22.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:13:22.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:13:25.822 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:13:25.823 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:13:55.824 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:13:55.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:13:58.882 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:13:58.883 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:14:28.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:14:29.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:14:32.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:14:32.159 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:15:02.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:15:02.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:15:05.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:15:05.214 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:15:35.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:15:35.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:15:38.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:15:38.270 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:16:08.272 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:16:08.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:16:11.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:16:11.327 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:16:41.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:16:41.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:16:44.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:16:44.383 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:17:14.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:17:14.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:17:17.438 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:17:17.438 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:17:47.440 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:17:47.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:17:50.498 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:17:50.499 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:18:20.501 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:18:20.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:18:23.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:18:23.582 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:18:53.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:18:53.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:18:56.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:18:56.639 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:19:26.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:19:26.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:19:29.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:19:29.694 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:19:59.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:19:59.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:20:02.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:20:02.751 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:20:32.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:20:32.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:20:35.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:20:35.838 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:21:05.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:21:05.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:21:08.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:21:08.895 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:21:38.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:21:38.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:21:41.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:21:41.951 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:22:11.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:22:11.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:22:15.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:22:15.007 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:22:45.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:22:45.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:22:48.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:22:48.063 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:23:18.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:23:18.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:23:21.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:23:21.118 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:23:51.119 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:23:51.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:23:54.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:23:54.175 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:24:24.177 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:24:24.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:24:27.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:24:27.230 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:24:57.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:24:57.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:25:00.286 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:25:00.287 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:25:30.288 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:25:30.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:25:33.346 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:25:33.347 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:26:03.348 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:26:03.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:26:06.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:26:06.431 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:26:36.432 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:26:36.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:26:39.490 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:26:39.491 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:27:09.494 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:27:09.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:27:12.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:27:12.574 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:27:42.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:27:42.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:27:45.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:27:45.631 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:28:15.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:28:15.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:28:18.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:28:18.719 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:28:48.721 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:28:48.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:28:51.779 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:28:51.780 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:29:21.784 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:29:21.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:29:24.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:29:24.863 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:29:54.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:29:54.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:29:57.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:29:57.951 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:30:27.953 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:30:27.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:30:31.010 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:30:31.011 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:31:01.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:31:01.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:31:04.098 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:31:04.099 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:31:34.101 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:31:34.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:31:37.182 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:31:37.183 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:32:07.184 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:32:07.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:32:10.238 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:32:10.239 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:32:40.241 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:32:40.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:32:43.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:32:43.327 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:33:13.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:33:13.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:33:16.386 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:33:16.387 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:33:46.388 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:33:46.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:33:49.471 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:33:49.472 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:34:19.474 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:34:19.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:34:22.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:34:22.526 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:34:52.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:34:52.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:34:55.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:34:55.583 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:35:25.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:35:25.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:35:28.639 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:35:28.640 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:35:58.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:35:58.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:36:01.726 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:36:01.727 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:36:31.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:36:31.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:36:34.782 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:36:34.783 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:37:04.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:37:04.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:37:07.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:37:07.839 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:37:37.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:37:37.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:37:40.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:37:40.895 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:38:10.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:38:10.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:38:13.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:38:13.950 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:38:43.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:38:43.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:38:47.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:38:47.007 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:39:17.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:39:17.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:39:20.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:39:20.062 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:39:50.065 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:39:50.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:39:53.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:39:53.119 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:40:23.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:40:23.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:40:26.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:40:26.175 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:40:56.177 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:40:56.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:40:59.262 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:40:59.263 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:41:29.266 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:41:29.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:41:32.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:41:32.351 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:42:02.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:42:02.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:42:05.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:42:05.407 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:42:35.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:42:35.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:42:38.494 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:42:38.495 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:43:08.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:43:08.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:43:11.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:43:11.551 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:43:41.553 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:43:41.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:43:44.606 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:43:44.607 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:44:14.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:44:14.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:44:17.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:44:17.663 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:44:47.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:44:47.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:44:50.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:44:50.719 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:45:20.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:45:20.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:45:23.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:45:23.775 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:45:53.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:45:53.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:45:56.830 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:45:56.850 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:46:26.852 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:46:26.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:46:29.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:46:29.919 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:46:59.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:46:59.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:47:02.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:47:02.975 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:47:32.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:47:32.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:47:36.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:47:36.031 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:48:06.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:48:06.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:48:09.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:48:09.086 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:48:39.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:48:39.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:48:42.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:48:42.143 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:49:12.143 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:49:12.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:49:15.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:49:15.198 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:49:45.200 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:49:45.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:49:48.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:49:48.255 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:50:18.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:50:18.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:50:21.314 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:50:21.315 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:50:51.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:50:51.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:50:54.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:50:54.453 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:51:24.455 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:51:24.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:51:27.522 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:51:27.522 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:51:57.524 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:51:57.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:52:00.606 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:52:00.607 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:52:30.609 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:52:30.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:52:33.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:52:33.663 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:53:03.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:53:03.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:53:06.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:53:06.718 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:53:36.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:53:36.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:53:39.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:53:39.775 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:54:09.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:54:09.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:54:12.830 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:54:12.831 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:54:42.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:54:42.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:54:45.891 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:54:45.892 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:55:15.894 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:55:15.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:55:18.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:55:18.975 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:55:48.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:55:48.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:55:52.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:55:52.031 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:56:22.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:56:22.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:56:25.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:56:25.086 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:56:55.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:56:55.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:56:58.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:56:58.143 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:57:28.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:57:28.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:57:31.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:57:31.199 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:58:01.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:58:01.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:58:04.286 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:58:04.287 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:58:34.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:58:34.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:58:37.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:58:37.343 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:59:07.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:59:07.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:59:10.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:59:10.399 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T06:59:40.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T06:59:40.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T06:59:43.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T06:59:43.487 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:00:13.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:00:13.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:00:16.546 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:00:16.547 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:00:46.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:00:46.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:00:49.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:00:49.631 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:01:19.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:01:19.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:01:22.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:01:22.687 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:01:52.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:01:52.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:01:55.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:01:55.743 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:02:25.745 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:02:25.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:02:28.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:02:28.799 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:02:58.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:02:58.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:03:01.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:03:01.855 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:03:31.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:03:31.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:03:34.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:03:34.911 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:04:04.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:04:04.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:04:07.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:04:07.966 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:04:37.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:04:37.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:04:41.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:04:41.023 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:05:11.025 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:05:11.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:05:14.078 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:05:14.078 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:05:44.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:05:44.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:05:47.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:05:47.135 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:06:17.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:06:17.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:06:20.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:06:20.191 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:06:50.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:06:50.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:06:53.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:06:53.279 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:07:23.281 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:07:23.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:07:26.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:07:26.335 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:07:56.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:07:56.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:07:59.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:07:59.391 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:08:29.393 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:08:29.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:08:32.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:08:32.447 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:09:02.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:09:02.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:09:05.502 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:09:05.502 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:09:35.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:09:35.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:09:38.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:09:38.559 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:10:08.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:10:08.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:10:11.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:10:11.615 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:10:41.617 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:10:41.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:10:44.670 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:10:44.671 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:11:14.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:11:14.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:11:17.726 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:11:17.727 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:11:47.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:11:47.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:11:50.782 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:11:50.783 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:12:20.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:12:20.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:12:23.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:12:23.838 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:12:53.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:12:53.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:12:56.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:12:56.895 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:13:26.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:13:26.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:13:29.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:13:29.951 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:13:59.953 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:13:59.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:14:03.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:14:03.007 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:14:33.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:14:33.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:14:36.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:14:36.063 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:15:06.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:15:06.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:15:09.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:15:09.119 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:15:39.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:15:39.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:15:42.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:15:42.207 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:16:12.209 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:16:12.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:16:15.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:16:15.295 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:16:45.296 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:16:45.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:16:48.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:16:48.351 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:17:18.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:17:18.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:17:21.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:17:21.406 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:17:51.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:17:51.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:17:54.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:17:54.463 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:18:24.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:18:24.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:18:27.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:18:27.519 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:18:57.521 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:18:57.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:19:00.606 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:19:00.607 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:19:30.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:19:30.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:19:33.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:19:33.663 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:20:03.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:20:03.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:20:06.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:20:06.719 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:20:36.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:20:36.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:20:39.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:20:39.775 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:21:09.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:21:09.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:21:12.830 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:21:12.831 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:21:42.833 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:21:42.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:21:45.886 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:21:45.887 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:22:15.888 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:22:15.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:22:18.942 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:22:18.943 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:22:48.945 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:22:48.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:22:52.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:22:52.031 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:23:22.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:23:22.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:23:25.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:23:25.087 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:23:55.089 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:23:55.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:23:58.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:23:58.175 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:24:28.177 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:24:28.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:24:31.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:24:31.231 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:25:01.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:25:01.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:25:04.286 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:25:04.286 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:25:34.288 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:25:34.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:25:37.374 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:25:37.375 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:26:07.377 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:26:07.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:26:10.440 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:26:10.441 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:26:40.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:26:40.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:26:43.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:26:43.519 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:27:13.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:27:13.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:27:16.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:27:16.574 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:27:46.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:27:46.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:27:49.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:27:49.631 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:28:19.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:28:19.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:28:22.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:28:22.718 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:28:52.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:28:52.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:28:55.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:28:55.776 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:29:25.778 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:29:25.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:29:28.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:29:28.863 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:29:58.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:29:58.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:30:01.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:30:01.918 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:30:31.921 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:30:31.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:30:34.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:30:34.975 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:31:04.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:31:04.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:31:08.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:31:08.031 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:31:38.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:31:38.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:31:41.090 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:31:41.091 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:32:11.093 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:32:11.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:32:14.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:32:14.175 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:32:44.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:32:44.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:32:47.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:32:47.231 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:33:17.233 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:33:17.236 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:33:20.286 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:33:20.287 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:33:50.288 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:33:50.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:33:53.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:33:53.343 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:34:23.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:34:23.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:34:26.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:34:26.430 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:34:56.433 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:34:56.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:34:59.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:34:59.519 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:35:29.521 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:35:29.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:35:32.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:35:32.575 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:36:02.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:36:02.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:36:05.634 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:36:05.635 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:36:35.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:36:35.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:36:38.722 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:36:38.723 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:37:08.726 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:37:08.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:37:11.806 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:37:11.807 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:37:41.808 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:37:41.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:37:44.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:37:44.863 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:38:14.865 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:38:14.868 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:38:17.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:38:17.951 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:38:47.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:38:47.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:38:51.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:38:51.007 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:39:21.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:39:21.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:39:24.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:39:24.063 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:39:54.065 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:39:54.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:39:57.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:39:57.119 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:40:27.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:40:27.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:40:30.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:40:30.207 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:41:00.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:41:00.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:41:03.266 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:41:03.267 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:41:33.269 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:41:33.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:41:36.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:41:36.351 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:42:06.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:42:06.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:42:09.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:42:09.407 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:42:39.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:42:39.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:42:42.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:42:42.462 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:43:12.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:43:12.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:43:15.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:43:15.519 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:43:45.521 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:43:45.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:43:48.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:43:48.575 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:44:18.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:44:18.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:44:21.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:44:21.663 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:44:51.665 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:44:51.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:44:54.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:44:54.719 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:45:24.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:45:24.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:45:27.778 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:45:27.779 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:45:57.781 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:45:57.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:46:00.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:46:00.863 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:46:30.865 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:46:30.868 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:46:33.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:46:33.952 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:47:03.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:47:03.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:47:07.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:47:07.039 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:47:37.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:47:37.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:47:40.126 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:47:40.127 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:48:10.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:48:10.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:48:13.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:48:13.215 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:48:43.217 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:48:43.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:48:46.277 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:48:46.277 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:49:16.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:49:16.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:49:19.358 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:49:19.359 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:49:49.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:49:49.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:49:52.418 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:49:52.419 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:50:22.421 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:50:22.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:50:25.502 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:50:25.503 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:50:55.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:50:55.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:50:58.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:50:58.559 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:51:28.560 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:51:28.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:51:31.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:51:31.615 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:52:01.620 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:52:01.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:52:04.702 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:52:04.703 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:52:34.705 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:52:34.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:52:37.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:52:37.759 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:53:07.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:53:07.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:53:10.818 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:53:10.819 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:53:40.820 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:53:40.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:53:43.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:53:43.903 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:54:13.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:54:13.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:54:16.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:54:16.958 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:54:46.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:54:46.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:54:50.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:54:50.047 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:55:20.049 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:55:20.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:55:23.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:55:23.103 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:55:53.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:55:53.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:55:56.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:55:56.159 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:56:26.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:56:26.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:56:29.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:56:29.215 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:56:59.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:56:59.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:57:02.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:57:02.276 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:57:32.278 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:57:32.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:57:35.358 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:57:35.358 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:58:05.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:58:05.362 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:58:08.414 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:58:08.415 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:58:38.419 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:58:38.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:58:41.502 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:58:41.503 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:59:11.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:59:11.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:59:14.562 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:59:14.563 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T07:59:44.565 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T07:59:44.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T07:59:47.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T07:59:47.647 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:00:17.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:00:17.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:00:20.702 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:00:20.703 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:00:50.705 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:00:50.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:00:53.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:00:53.759 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:01:23.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:01:23.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:01:26.818 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:01:26.820 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:01:56.821 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:01:56.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:01:59.906 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:01:59.906 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:02:29.909 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:02:29.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:02:32.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:02:32.991 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:03:02.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:03:02.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:03:06.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:03:06.047 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:03:36.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:03:36.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:03:39.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:03:39.103 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:04:09.105 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:04:09.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:04:12.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:04:12.159 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:04:42.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:04:42.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:04:45.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:04:45.215 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:05:15.217 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:05:15.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:05:18.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:05:18.271 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:05:48.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:05:48.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:05:51.333 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:05:51.334 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:06:21.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:06:21.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:06:24.414 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:06:24.415 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:06:54.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:06:54.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:06:57.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:06:57.471 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:07:27.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:07:27.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:07:30.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:07:30.559 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:08:00.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:08:00.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:08:03.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:08:03.615 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:08:33.617 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:08:33.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:08:36.674 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:08:36.675 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:09:06.676 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:09:06.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:09:09.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:09:09.759 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:09:39.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:09:39.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:09:42.814 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:09:42.815 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:10:12.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:10:12.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:10:15.870 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:10:15.871 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:10:45.873 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:10:45.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:10:48.930 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:10:48.930 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:11:18.932 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:11:18.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:11:22.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:11:22.015 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:11:52.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:11:52.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:11:55.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:11:55.070 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:12:25.073 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:12:25.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:12:28.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:12:28.159 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:12:58.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:12:58.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:13:01.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:13:01.215 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:13:31.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:13:31.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:13:34.302 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:13:34.303 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:14:04.303 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:14:04.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:14:07.358 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:14:07.359 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:14:37.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:14:37.362 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:14:40.414 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:14:40.415 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:15:10.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:15:10.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:15:13.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:15:13.471 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:15:43.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:15:43.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:15:46.535 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:15:46.536 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:16:16.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:16:16.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:16:19.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:16:19.615 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:16:49.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:16:49.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:16:52.670 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:16:52.671 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:17:22.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:17:22.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:17:25.762 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:17:25.763 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:17:55.765 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:17:55.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:17:58.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:17:58.847 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:18:28.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:18:28.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:18:31.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:18:31.903 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:19:01.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:19:01.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:19:04.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:19:04.991 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:19:34.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:19:34.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:19:38.078 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:19:38.079 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:20:08.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:20:08.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:20:11.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:20:11.135 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:20:41.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:20:41.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:20:44.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:20:44.191 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:21:14.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:21:14.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:21:17.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:21:17.246 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:21:47.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:21:47.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:21:50.345 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:21:50.346 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:22:20.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:22:20.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:22:23.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:22:23.423 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:22:53.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:22:53.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:22:56.478 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:22:56.478 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:23:26.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:23:26.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:23:29.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:23:29.534 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:23:59.536 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:23:59.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:24:02.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:24:02.590 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:24:32.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:24:32.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:24:35.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:24:35.646 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:25:05.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:25:05.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:25:08.702 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:25:08.702 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:25:38.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:25:38.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:25:41.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:25:41.759 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:26:11.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:26:11.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:26:14.814 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:26:14.814 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:26:44.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:26:44.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:26:47.870 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:26:47.871 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:27:17.873 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:27:17.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:27:20.962 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:27:20.963 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:27:50.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:27:50.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:27:54.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:27:54.046 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:28:24.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:28:24.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:28:27.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:28:27.103 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:28:57.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:28:57.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:29:00.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:29:00.190 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:29:30.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:29:30.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:29:33.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:29:33.247 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:30:03.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:30:03.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:30:06.302 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:30:06.303 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:30:36.304 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:30:36.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:30:39.358 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:30:39.358 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:31:09.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:31:09.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:31:12.414 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:31:12.414 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:31:42.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:31:42.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:31:45.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:31:45.470 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:32:15.472 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:32:15.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:32:18.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:32:18.527 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:32:48.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:32:48.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:32:51.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:32:51.582 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:33:21.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:33:21.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:33:24.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:33:24.639 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:33:54.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:33:54.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:33:57.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:33:57.695 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:34:27.697 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:34:27.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:34:30.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:34:30.751 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:35:00.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:35:00.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:35:03.806 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:35:03.806 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:35:33.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:35:33.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:35:36.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:35:36.895 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:36:06.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:36:06.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:36:09.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:36:09.950 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:36:39.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:36:39.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:36:43.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:36:43.007 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:37:13.009 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:37:13.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:37:16.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:37:16.063 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:37:46.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:37:46.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:37:49.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:37:49.119 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:38:19.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:38:19.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:38:22.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:38:22.207 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:38:52.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:38:52.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:38:55.262 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:38:55.263 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:39:25.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:39:25.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:39:28.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:39:28.318 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:39:58.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:39:58.322 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:40:01.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:40:01.407 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:40:31.409 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:40:31.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:40:34.494 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:40:34.495 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:41:04.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:41:04.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:41:07.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:41:07.582 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:41:37.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:41:37.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:41:40.670 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:41:40.671 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:42:10.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:42:10.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:42:13.726 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:42:13.727 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:42:43.729 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:42:43.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:42:46.782 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:42:46.782 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:43:16.784 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:43:16.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:43:19.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:43:19.839 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:43:49.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:43:49.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:43:52.898 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:43:52.899 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:44:22.900 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:44:22.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:44:25.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:44:25.982 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:44:55.984 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:44:55.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:44:59.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:44:59.039 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:45:29.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:45:29.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:45:32.094 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:45:32.095 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:46:02.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:46:02.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:46:05.182 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:46:05.183 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:46:35.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:46:35.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:46:38.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:46:38.271 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:47:08.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:47:08.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:47:11.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:47:11.327 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:47:41.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:47:41.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:47:44.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:47:44.383 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:48:14.385 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:48:14.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:48:17.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:48:17.471 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:48:47.472 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:48:47.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:48:50.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:48:50.527 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:49:20.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:49:20.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:49:23.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:49:23.583 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:49:53.585 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:49:53.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:49:56.674 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:49:56.675 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:50:26.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:50:26.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:50:29.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:50:29.759 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:50:59.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:50:59.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:51:02.814 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:51:02.815 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:51:32.817 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:51:32.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:51:35.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:51:35.903 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:52:05.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:52:05.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:52:08.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:52:08.959 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:52:38.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:52:38.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:52:42.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:52:42.047 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:53:12.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:53:12.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:53:15.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:53:15.103 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:53:45.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:53:45.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:53:48.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:53:48.159 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:54:18.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:54:18.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:54:21.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:54:21.214 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:54:51.217 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:54:51.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:54:54.306 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:54:54.307 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:55:24.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:55:24.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:55:27.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:55:27.391 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:55:57.393 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:55:57.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:56:00.478 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:56:00.479 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:56:30.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:56:30.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:56:33.566 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:56:33.567 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:57:03.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:57:03.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:57:06.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:57:06.623 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:57:36.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:57:36.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:57:39.682 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:57:39.683 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:58:09.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:58:09.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:58:12.770 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:58:12.770 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:58:42.773 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:58:42.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:58:45.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:58:45.855 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:59:15.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:59:15.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:59:18.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:59:18.911 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T08:59:48.913 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T08:59:48.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T08:59:51.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T08:59:51.966 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:00:21.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:00:21.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:00:25.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:00:25.023 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:00:55.024 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:00:55.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:00:58.078 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:00:58.078 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:01:28.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:01:28.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:01:31.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:01:31.135 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:02:01.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:02:01.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:02:04.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:02:04.191 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:02:34.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:02:34.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:02:37.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:02:37.247 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:03:07.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:03:07.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:03:10.312 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:03:10.313 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:03:40.315 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:03:40.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:03:43.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:03:43.391 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:04:13.393 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:04:13.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:04:16.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:04:16.447 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:04:46.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:04:46.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:04:49.502 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:04:49.503 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:05:19.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:05:19.508 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:05:22.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:05:22.591 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:05:52.593 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:05:52.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:05:55.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:05:55.647 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:06:25.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:06:25.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:06:28.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:06:28.734 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:06:58.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:06:58.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:07:01.790 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:07:01.791 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:07:31.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:07:31.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:07:34.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:07:34.847 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:08:04.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:08:04.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:08:07.906 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:08:07.907 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:08:37.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:08:37.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:08:40.994 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:08:40.995 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:09:10.996 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:09:10.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:09:14.082 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:09:14.083 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:09:44.084 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:09:44.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:09:47.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:09:47.167 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:10:17.169 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:10:17.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:10:20.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:10:20.254 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:10:50.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-19T09:10:50.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-19T09:10:53.310 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2020-05-19T09:10:53.311 DEBUG:tasks.ceph:Missed logrotate, node 'smithi080' is offline 2020-05-19T09:11:15.271 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-05-19T09:11:15.273 DEBUG:teuthology.task.console_log:Killing console logger for smithi196 2020-05-19T09:11:15.274 DEBUG:teuthology.task.console_log:Killing console logger for smithi080 2020-05-19T09:11:15.274 DEBUG:teuthology.task.console_log:Killing console logger for smithi196 2020-05-19T09:11:15.275 DEBUG:teuthology.task.console_log:Killing console logger for smithi080 2020-05-19T09:11:15.275 DEBUG:teuthology.exit:Finished running handlers